Jun 10 12:48:58.143429 ip-10-0-128-230 systemd[1]: Starting Kubernetes Kubelet... Jun 10 12:48:58.552456 ip-10-0-128-230 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 10 12:48:58.552456 ip-10-0-128-230 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 10 12:48:58.552456 ip-10-0-128-230 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 10 12:48:58.552456 ip-10-0-128-230 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 10 12:48:58.552456 ip-10-0-128-230 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 10 12:48:58.553496 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.553413 2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 10 12:48:58.557357 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557342 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 12:48:58.557357 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557356 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 12:48:58.557419 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557360 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 12:48:58.557419 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557364 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 12:48:58.557419 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557367 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 12:48:58.557419 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557370 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 12:48:58.557419 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557379 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 12:48:58.557419 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557382 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 12:48:58.557419 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557385 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 12:48:58.557419 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557387 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 12:48:58.557419 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557390 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 12:48:58.557419 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557392 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 12:48:58.557419 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557395 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 12:48:58.557419 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557397 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 12:48:58.557419 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557400 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 12:48:58.557419 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557402 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 12:48:58.557419 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557405 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 12:48:58.557419 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557407 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 12:48:58.557419 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557410 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 12:48:58.557419 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557412 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 12:48:58.557419 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557415 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 12:48:58.557419 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557417 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 12:48:58.557901 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557419 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 12:48:58.557901 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557424 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 12:48:58.557901 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557428 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 12:48:58.557901 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557431 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 12:48:58.557901 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557434 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 12:48:58.557901 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557438 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 12:48:58.557901 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557442 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 12:48:58.557901 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557444 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 12:48:58.557901 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557447 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 12:48:58.557901 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557450 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 12:48:58.557901 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557452 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 12:48:58.557901 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557455 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 12:48:58.557901 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557457 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 12:48:58.557901 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557460 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 12:48:58.557901 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557462 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 12:48:58.557901 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557464 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 12:48:58.557901 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557467 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 12:48:58.557901 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557470 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 12:48:58.557901 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557472 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 12:48:58.558416 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557476 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 12:48:58.558416 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557478 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 12:48:58.558416 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557481 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 12:48:58.558416 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557483 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 12:48:58.558416 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557485 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 12:48:58.558416 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557488 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 12:48:58.558416 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557490 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 12:48:58.558416 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557493 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 12:48:58.558416 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557495 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 12:48:58.558416 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557497 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 12:48:58.558416 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557500 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 12:48:58.558416 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557502 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 12:48:58.558416 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557504 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 12:48:58.558416 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557507 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 12:48:58.558416 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557510 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 12:48:58.558416 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557513 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 12:48:58.558416 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557516 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 12:48:58.558416 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557519 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 12:48:58.558416 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557521 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 12:48:58.558416 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557524 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 12:48:58.558891 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557526 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 12:48:58.558891 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557529 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 12:48:58.558891 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557531 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 12:48:58.558891 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557533 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 12:48:58.558891 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557536 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 12:48:58.558891 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557538 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 12:48:58.558891 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557540 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 12:48:58.558891 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557543 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 12:48:58.558891 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557546 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 12:48:58.558891 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557548 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 12:48:58.558891 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557551 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 12:48:58.558891 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557554 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 12:48:58.558891 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557556 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 12:48:58.558891 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557560 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 12:48:58.558891 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557562 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 12:48:58.558891 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557564 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 12:48:58.558891 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557567 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 12:48:58.558891 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557569 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 12:48:58.558891 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557572 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 12:48:58.559355 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557575 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 12:48:58.559355 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557578 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 12:48:58.559355 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557581 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 12:48:58.559355 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557584 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 12:48:58.559355 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557587 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 12:48:58.559355 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557589 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 12:48:58.559355 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557592 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 12:48:58.559355 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557596 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 12:48:58.559355 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557599 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 12:48:58.559355 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557601 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 12:48:58.559355 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557603 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 12:48:58.559355 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557606 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 12:48:58.559355 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557609 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 12:48:58.559355 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557611 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 12:48:58.559355 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557613 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 12:48:58.559355 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557616 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 12:48:58.559355 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557618 2581 feature_gate.go:328] unrecognized feature gate: Example Jun 10 12:48:58.559355 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557621 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 12:48:58.559355 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.557623 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 12:48:58.559355 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559305 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 12:48:58.559887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559313 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 12:48:58.559887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559316 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 12:48:58.559887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559319 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 12:48:58.559887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559322 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 12:48:58.559887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559326 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 12:48:58.559887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559329 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 12:48:58.559887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559331 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 12:48:58.559887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559334 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 12:48:58.559887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559337 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 12:48:58.559887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559340 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 12:48:58.559887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559342 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 12:48:58.559887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559344 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 12:48:58.559887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559347 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 12:48:58.559887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559349 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 12:48:58.559887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559352 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 12:48:58.559887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559354 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 12:48:58.559887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559357 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 12:48:58.559887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559360 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 12:48:58.559887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559362 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 12:48:58.559887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559365 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 12:48:58.560430 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559368 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 12:48:58.560430 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559371 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 12:48:58.560430 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559373 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 12:48:58.560430 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559376 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 12:48:58.560430 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559379 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 12:48:58.560430 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559383 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 12:48:58.560430 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559386 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 12:48:58.560430 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559389 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 12:48:58.560430 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559392 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 12:48:58.560430 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559394 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 12:48:58.560430 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559396 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 12:48:58.560430 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559399 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 12:48:58.560430 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559401 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 12:48:58.560430 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559404 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 12:48:58.560430 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559408 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 12:48:58.560430 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559412 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 12:48:58.560430 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559415 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 12:48:58.560430 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559418 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 12:48:58.560430 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559421 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 12:48:58.560895 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559424 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 12:48:58.560895 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559427 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 12:48:58.560895 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559429 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 12:48:58.560895 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559431 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 12:48:58.560895 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559434 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 12:48:58.560895 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559436 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 12:48:58.560895 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559439 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 12:48:58.560895 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559441 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 12:48:58.560895 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559444 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 12:48:58.560895 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559446 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 12:48:58.560895 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559450 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 12:48:58.560895 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559453 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 12:48:58.560895 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559456 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 12:48:58.560895 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559458 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 12:48:58.560895 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559461 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 12:48:58.560895 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559463 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 12:48:58.560895 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559466 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 12:48:58.560895 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559468 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 12:48:58.560895 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559471 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 12:48:58.561392 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559473 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 12:48:58.561392 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559475 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 12:48:58.561392 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559478 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 12:48:58.561392 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559481 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 12:48:58.561392 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559483 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 12:48:58.561392 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559486 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 12:48:58.561392 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559488 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 12:48:58.561392 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559492 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 12:48:58.561392 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559494 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 12:48:58.561392 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559497 2581 feature_gate.go:328] unrecognized feature gate: Example Jun 10 12:48:58.561392 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559499 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 12:48:58.561392 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559502 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 12:48:58.561392 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559505 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 12:48:58.561392 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559508 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 12:48:58.561392 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559510 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 12:48:58.561392 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559512 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 12:48:58.561392 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559515 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 12:48:58.561392 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559517 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 12:48:58.561392 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559520 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 12:48:58.561392 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559523 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 12:48:58.561876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559525 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 12:48:58.561876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559528 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 12:48:58.561876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559530 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 12:48:58.561876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559533 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 12:48:58.561876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559535 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 12:48:58.561876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559538 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 12:48:58.561876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559540 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 12:48:58.561876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559543 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 12:48:58.561876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559545 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 12:48:58.561876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559547 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 12:48:58.561876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559550 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 12:48:58.561876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559553 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 12:48:58.561876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559555 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 12:48:58.561876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559558 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 12:48:58.561876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559560 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 12:48:58.561876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559563 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 12:48:58.561876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559565 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 12:48:58.561876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559568 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 12:48:58.561876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559570 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 12:48:58.561876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559573 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 12:48:58.562380 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559950 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 12:48:58.562380 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559955 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 12:48:58.562380 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559958 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 12:48:58.562380 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559961 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 12:48:58.562380 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559964 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 12:48:58.562380 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559967 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 12:48:58.562380 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559969 2581 feature_gate.go:328] unrecognized feature gate: Example Jun 10 12:48:58.562380 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559972 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 12:48:58.562380 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559974 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 12:48:58.562380 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559976 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 12:48:58.562380 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559979 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 12:48:58.562380 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559981 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 12:48:58.562380 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559984 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 12:48:58.562380 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559987 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 12:48:58.562380 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559989 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 12:48:58.562380 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559992 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 12:48:58.562380 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559994 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 12:48:58.562380 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559996 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 12:48:58.562380 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.559999 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 12:48:58.562380 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560002 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 12:48:58.562876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560004 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 12:48:58.562876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560007 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 12:48:58.562876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560009 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 12:48:58.562876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560012 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 12:48:58.562876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560014 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 12:48:58.562876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560017 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 12:48:58.562876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560020 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 12:48:58.562876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560023 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 12:48:58.562876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560025 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 12:48:58.562876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560028 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 12:48:58.562876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560030 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 12:48:58.562876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560033 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 12:48:58.562876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560036 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 12:48:58.562876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560039 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 12:48:58.562876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560041 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 12:48:58.562876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560044 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 12:48:58.562876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560046 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 12:48:58.562876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560049 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 12:48:58.562876 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560052 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 12:48:58.563364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560054 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 12:48:58.563364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560057 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 12:48:58.563364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560059 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 12:48:58.563364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560062 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 12:48:58.563364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560064 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 12:48:58.563364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560067 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 12:48:58.563364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560069 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 12:48:58.563364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560072 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 12:48:58.563364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560074 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 12:48:58.563364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560077 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 12:48:58.563364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560079 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 12:48:58.563364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560082 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 12:48:58.563364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560084 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 12:48:58.563364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560086 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 12:48:58.563364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560089 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 12:48:58.563364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560091 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 12:48:58.563364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560094 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 12:48:58.563364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560096 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 12:48:58.563364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560099 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 12:48:58.563364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560103 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 12:48:58.563887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560106 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 12:48:58.563887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560109 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 12:48:58.563887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560113 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 12:48:58.563887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560116 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 12:48:58.563887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560120 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 12:48:58.563887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560123 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 12:48:58.563887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560125 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 12:48:58.563887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560128 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 12:48:58.563887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560131 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 12:48:58.563887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560134 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 12:48:58.563887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560136 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 12:48:58.563887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560139 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 12:48:58.563887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560142 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 12:48:58.563887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560144 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 12:48:58.563887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560147 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 12:48:58.563887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560149 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 12:48:58.563887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560151 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 12:48:58.563887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560154 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 12:48:58.563887 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560156 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 12:48:58.564373 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560159 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 12:48:58.564373 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560161 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 12:48:58.564373 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560177 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 12:48:58.564373 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560180 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 12:48:58.564373 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560182 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 12:48:58.564373 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560185 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 12:48:58.564373 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560198 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 12:48:58.564373 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560202 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 12:48:58.564373 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560206 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 12:48:58.564373 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560208 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 12:48:58.564373 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560211 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 12:48:58.564373 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560214 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 12:48:58.564373 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560217 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 12:48:58.564373 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560219 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 12:48:58.564373 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560223 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 12:48:58.564373 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560225 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 12:48:58.564373 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560227 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 12:48:58.564373 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560230 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 12:48:58.564373 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560232 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 12:48:58.564373 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560235 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560237 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560311 2581 flags.go:64] FLAG: --address="0.0.0.0" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560318 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560324 2581 flags.go:64] FLAG: --anonymous-auth="true" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560328 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560333 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560336 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560340 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560344 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560347 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560350 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560354 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560357 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560360 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560363 2581 flags.go:64] FLAG: --cgroup-root="" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560366 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560369 2581 flags.go:64] FLAG: --client-ca-file="" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560371 2581 flags.go:64] FLAG: --cloud-provider="external" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560374 2581 flags.go:64] FLAG: --cluster-dns="[]" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560379 2581 flags.go:64] FLAG: --cluster-domain="" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560381 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560385 2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560388 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 10 12:48:58.564868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560391 2581 flags.go:64] FLAG: --container-log-max-files="5" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560395 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560398 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560401 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560405 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560408 2581 flags.go:64] FLAG: --contention-profiling="false" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560411 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560414 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560417 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560419 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560423 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560426 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560429 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560432 2581 flags.go:64] FLAG: --enable-load-reader="false" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560434 2581 flags.go:64] FLAG: --enable-server="true" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560438 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560443 2581 flags.go:64] FLAG: --event-burst="100" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560446 2581 flags.go:64] FLAG: --event-qps="50" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560448 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560451 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560454 2581 flags.go:64] FLAG: --eviction-hard="" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560458 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560461 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560463 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560466 2581 flags.go:64] FLAG: --eviction-soft="" Jun 10 12:48:58.565460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560469 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560472 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560475 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560482 2581 flags.go:64] FLAG: --experimental-mounter-path="" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560484 2581 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560487 2581 flags.go:64] FLAG: --fail-swap-on="true" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560490 2581 flags.go:64] FLAG: --feature-gates="" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560493 2581 flags.go:64] FLAG: --file-check-frequency="20s" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560496 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560501 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560504 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560507 2581 flags.go:64] FLAG: --healthz-port="10248" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560510 2581 flags.go:64] FLAG: --help="false" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560512 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-128-230.ec2.internal" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560515 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560518 2581 flags.go:64] FLAG: --http-check-frequency="20s" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560521 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560524 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560527 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560530 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560533 2581 flags.go:64] FLAG: --image-service-endpoint="" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560536 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560538 2581 flags.go:64] FLAG: --kube-api-burst="100" Jun 10 12:48:58.566065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560541 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560544 2581 flags.go:64] FLAG: --kube-api-qps="50" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560547 2581 flags.go:64] FLAG: --kube-reserved="" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560550 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560552 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560555 2581 flags.go:64] FLAG: --kubelet-cgroups="" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560558 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560560 2581 flags.go:64] FLAG: --lock-file="" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560563 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560566 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560568 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560573 2581 flags.go:64] FLAG: --log-json-split-stream="false" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560577 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560580 2581 flags.go:64] FLAG: --log-text-split-stream="false" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560582 2581 flags.go:64] FLAG: --logging-format="text" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560585 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560588 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560591 2581 flags.go:64] FLAG: --manifest-url="" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560595 2581 flags.go:64] FLAG: --manifest-url-header="" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560599 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560602 2581 flags.go:64] FLAG: --max-open-files="1000000" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560606 2581 flags.go:64] FLAG: --max-pods="110" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560609 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560611 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560614 2581 flags.go:64] FLAG: --memory-manager-policy="None" Jun 10 12:48:58.566680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560617 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560620 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560623 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560625 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560632 2581 flags.go:64] FLAG: --node-status-max-images="50" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560635 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560638 2581 flags.go:64] FLAG: --oom-score-adj="-999" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560641 2581 flags.go:64] FLAG: --pod-cidr="" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560644 2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560649 2581 flags.go:64] FLAG: --pod-manifest-path="" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560652 2581 flags.go:64] FLAG: --pod-max-pids="-1" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560655 2581 flags.go:64] FLAG: --pods-per-core="0" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560657 2581 flags.go:64] FLAG: --port="10250" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560660 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560663 2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fe653f6fedfde66e" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560666 2581 flags.go:64] FLAG: --qos-reserved="" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560669 2581 flags.go:64] FLAG: --read-only-port="10255" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560671 2581 flags.go:64] FLAG: --register-node="true" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560674 2581 flags.go:64] FLAG: --register-with-taints="" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560678 2581 flags.go:64] FLAG: --registry-burst="10" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560682 2581 flags.go:64] FLAG: --registry-qps="5" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560684 2581 flags.go:64] FLAG: --reserved-cpus="" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560687 2581 flags.go:64] FLAG: --reserved-memory="" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560690 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560693 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 10 12:48:58.567543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560698 2581 flags.go:64] FLAG: --rotate-certificates="false" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560701 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560703 2581 flags.go:64] FLAG: --runonce="false" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560706 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560709 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560712 2581 flags.go:64] FLAG: --seccomp-default="false" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560714 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560717 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560720 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560723 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560726 2581 flags.go:64] FLAG: --storage-driver-password="root" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560729 2581 flags.go:64] FLAG: --storage-driver-secure="false" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560731 2581 flags.go:64] FLAG: --storage-driver-table="stats" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560734 2581 flags.go:64] FLAG: --storage-driver-user="root" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560737 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560739 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560743 2581 flags.go:64] FLAG: --system-cgroups="" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560746 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560751 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560753 2581 flags.go:64] FLAG: --tls-cert-file="" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560756 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560760 2581 flags.go:64] FLAG: --tls-min-version="" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560763 2581 flags.go:64] FLAG: --tls-private-key-file="" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560765 2581 flags.go:64] FLAG: --topology-manager-policy="none" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560768 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 10 12:48:58.568631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560771 2581 flags.go:64] FLAG: --topology-manager-scope="container" Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560773 2581 flags.go:64] FLAG: --v="2" Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560778 2581 flags.go:64] FLAG: --version="false" Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560782 2581 flags.go:64] FLAG: --vmodule="" Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560786 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.560789 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560898 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560903 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560906 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560909 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560912 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560914 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560917 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560919 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560922 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560924 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560927 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560929 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560932 2581 feature_gate.go:328] unrecognized feature gate: Example Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560934 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560938 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 12:48:58.569662 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560942 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 12:48:58.570259 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560945 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 12:48:58.570259 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560947 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 12:48:58.570259 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560950 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 12:48:58.570259 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560952 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 12:48:58.570259 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560955 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 12:48:58.570259 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560959 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 12:48:58.570259 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560962 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 12:48:58.570259 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560965 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 12:48:58.570259 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560968 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 12:48:58.570259 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560970 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 12:48:58.570259 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.560988 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 12:48:58.570259 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561012 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 12:48:58.570259 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561018 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 12:48:58.570259 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561021 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 12:48:58.570259 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561024 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 12:48:58.570259 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561027 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 12:48:58.570259 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561030 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 12:48:58.570259 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561033 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 12:48:58.570259 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561036 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 12:48:58.570259 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561038 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 12:48:58.570993 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561041 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 12:48:58.570993 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561044 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 12:48:58.570993 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561046 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 12:48:58.570993 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561049 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 12:48:58.570993 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561051 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 12:48:58.570993 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561054 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 12:48:58.570993 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561057 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 12:48:58.570993 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561059 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 12:48:58.570993 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561062 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 12:48:58.570993 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561065 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 12:48:58.570993 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561067 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 12:48:58.570993 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561070 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 12:48:58.570993 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561072 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 12:48:58.570993 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561075 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 12:48:58.570993 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561077 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 12:48:58.570993 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561079 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 12:48:58.570993 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561082 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 12:48:58.570993 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561085 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 12:48:58.570993 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561087 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 12:48:58.570993 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561090 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 12:48:58.571529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561092 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 12:48:58.571529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561094 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 12:48:58.571529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561097 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 12:48:58.571529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561101 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 12:48:58.571529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561104 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 12:48:58.571529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561107 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 12:48:58.571529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561109 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 12:48:58.571529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561112 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 12:48:58.571529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561114 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 12:48:58.571529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561117 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 12:48:58.571529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561120 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 12:48:58.571529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561122 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 12:48:58.571529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561124 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 12:48:58.571529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561127 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 12:48:58.571529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561130 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 12:48:58.571529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561132 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 12:48:58.571529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561135 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 12:48:58.571529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561138 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 12:48:58.571529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561140 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 12:48:58.571529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561143 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 12:48:58.572031 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561145 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 12:48:58.572031 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561147 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 12:48:58.572031 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561150 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 12:48:58.572031 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561154 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 12:48:58.572031 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561160 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 12:48:58.572031 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561180 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 12:48:58.572031 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561184 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 12:48:58.572031 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561186 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 12:48:58.572031 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561189 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 12:48:58.572031 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561191 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 12:48:58.572031 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561195 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 12:48:58.572031 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561197 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 12:48:58.572031 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561200 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 12:48:58.572031 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561202 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 12:48:58.572031 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561205 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 12:48:58.572031 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561212 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 12:48:58.572031 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561215 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 12:48:58.572031 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561218 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 12:48:58.572031 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561221 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 12:48:58.572606 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561223 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 12:48:58.572606 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561226 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 12:48:58.572606 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561229 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 12:48:58.572606 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.561231 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 12:48:58.572606 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.561922 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 10 12:48:58.572606 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.569289 2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 10 12:48:58.572606 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.569673 2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 10 12:48:58.572606 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.570535 2581 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 10 12:48:58.572606 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.570574 2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 10 12:48:58.572606 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570798 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 12:48:58.572606 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570806 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 12:48:58.572606 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570810 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 12:48:58.572606 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570813 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 12:48:58.572606 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570816 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 12:48:58.572606 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570819 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 12:48:58.572985 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570823 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 12:48:58.572985 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570826 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 12:48:58.572985 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570828 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 12:48:58.572985 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570831 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 12:48:58.572985 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570833 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 12:48:58.572985 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570836 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 12:48:58.572985 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570839 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 12:48:58.572985 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570842 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 12:48:58.572985 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570844 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 12:48:58.572985 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570848 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 12:48:58.572985 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570850 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 12:48:58.572985 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570854 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 12:48:58.572985 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570858 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 12:48:58.572985 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570861 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 12:48:58.572985 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570864 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 12:48:58.572985 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570866 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 12:48:58.572985 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570869 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 12:48:58.572985 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570871 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 12:48:58.572985 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570874 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 12:48:58.573472 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570876 2581 feature_gate.go:328] unrecognized feature gate: Example Jun 10 12:48:58.573472 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570879 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 12:48:58.573472 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570881 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 12:48:58.573472 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570884 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 12:48:58.573472 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570886 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 12:48:58.573472 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570890 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 12:48:58.573472 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570892 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 12:48:58.573472 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570896 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 12:48:58.573472 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570899 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 12:48:58.573472 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570902 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 12:48:58.573472 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570904 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 12:48:58.573472 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570906 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 12:48:58.573472 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570909 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 12:48:58.573472 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570912 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 12:48:58.573472 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570914 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 12:48:58.573472 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570917 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 12:48:58.573472 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570919 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 12:48:58.573472 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570921 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 12:48:58.573472 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570924 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 12:48:58.573472 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570926 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 12:48:58.573956 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570929 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 12:48:58.573956 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570931 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 12:48:58.573956 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570935 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 12:48:58.573956 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570938 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 12:48:58.573956 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570941 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 12:48:58.573956 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570943 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 12:48:58.573956 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570946 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 12:48:58.573956 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570948 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 12:48:58.573956 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570951 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 12:48:58.573956 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570953 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 12:48:58.573956 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570956 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 12:48:58.573956 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570958 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 12:48:58.573956 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570961 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 12:48:58.573956 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570963 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 12:48:58.573956 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570966 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 12:48:58.573956 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570968 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 12:48:58.573956 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570971 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 12:48:58.573956 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570973 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 12:48:58.573956 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570976 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 12:48:58.573956 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570979 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 12:48:58.574506 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570982 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 12:48:58.574506 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570985 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 12:48:58.574506 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570987 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 12:48:58.574506 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570990 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 12:48:58.574506 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570992 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 12:48:58.574506 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570995 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 12:48:58.574506 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.570997 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 12:48:58.574506 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571000 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 12:48:58.574506 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571002 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 12:48:58.574506 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571005 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 12:48:58.574506 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571009 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 12:48:58.574506 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571012 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 12:48:58.574506 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571015 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 12:48:58.574506 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571018 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 12:48:58.574506 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571021 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 12:48:58.574506 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571023 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 12:48:58.574506 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571026 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 12:48:58.574506 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571030 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 12:48:58.574506 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571032 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 12:48:58.574972 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571035 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 12:48:58.574972 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571038 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 12:48:58.574972 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571041 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 12:48:58.574972 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571043 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 12:48:58.574972 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571046 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 12:48:58.574972 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571049 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 12:48:58.574972 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571052 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 12:48:58.574972 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571055 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 12:48:58.574972 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571057 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 12:48:58.574972 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571060 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 12:48:58.574972 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571062 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 12:48:58.574972 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571065 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 12:48:58.574972 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571068 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 12:48:58.574972 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571071 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 12:48:58.574972 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571074 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 12:48:58.575364 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.571079 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 10 12:48:58.575364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571209 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 10 12:48:58.575364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571215 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 10 12:48:58.575364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571218 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 10 12:48:58.575364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571221 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 10 12:48:58.575364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571225 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 10 12:48:58.575364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571229 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 10 12:48:58.575364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571233 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 10 12:48:58.575364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571236 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 10 12:48:58.575364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571238 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 10 12:48:58.575364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571241 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 10 12:48:58.575364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571244 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 10 12:48:58.575364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571246 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 10 12:48:58.575364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571249 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 10 12:48:58.575718 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571253 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 10 12:48:58.575718 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571256 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 10 12:48:58.575718 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571258 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 10 12:48:58.575718 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571261 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 10 12:48:58.575718 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571263 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 10 12:48:58.575718 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571266 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 10 12:48:58.575718 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571268 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 10 12:48:58.575718 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571271 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 10 12:48:58.575718 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571274 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 10 12:48:58.575718 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571276 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 10 12:48:58.575718 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571279 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 10 12:48:58.575718 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571282 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 10 12:48:58.575718 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571284 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 10 12:48:58.575718 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571287 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 10 12:48:58.575718 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571289 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 10 12:48:58.575718 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571292 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 10 12:48:58.575718 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571295 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 10 12:48:58.575718 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571298 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 10 12:48:58.575718 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571300 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 10 12:48:58.575718 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571303 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 10 12:48:58.576218 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571305 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 10 12:48:58.576218 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571308 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 10 12:48:58.576218 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571311 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 10 12:48:58.576218 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571313 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 10 12:48:58.576218 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571316 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 10 12:48:58.576218 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571318 2581 feature_gate.go:328] unrecognized feature gate: Example Jun 10 12:48:58.576218 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571321 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 10 12:48:58.576218 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571323 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jun 10 12:48:58.576218 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571326 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 10 12:48:58.576218 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571328 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 10 12:48:58.576218 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571331 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 10 12:48:58.576218 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571333 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 10 12:48:58.576218 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571335 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 10 12:48:58.576218 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571338 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 10 12:48:58.576218 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571341 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 10 12:48:58.576218 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571344 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 10 12:48:58.576218 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571346 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 10 12:48:58.576218 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571348 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 10 12:48:58.576218 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571351 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 10 12:48:58.576218 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571353 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 10 12:48:58.576704 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571356 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 10 12:48:58.576704 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571358 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 10 12:48:58.576704 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571361 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 10 12:48:58.576704 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571364 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 10 12:48:58.576704 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571366 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 10 12:48:58.576704 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571368 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 10 12:48:58.576704 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571371 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 10 12:48:58.576704 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571374 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 10 12:48:58.576704 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571376 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 10 12:48:58.576704 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571379 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 10 12:48:58.576704 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571381 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 10 12:48:58.576704 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571383 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 10 12:48:58.576704 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571386 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 10 12:48:58.576704 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571388 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 10 12:48:58.576704 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571391 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 10 12:48:58.576704 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571394 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 10 12:48:58.576704 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571397 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 10 12:48:58.576704 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571400 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 10 12:48:58.576704 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571404 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 10 12:48:58.577156 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571408 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 10 12:48:58.577156 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571411 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 10 12:48:58.577156 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571413 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 10 12:48:58.577156 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571416 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 10 12:48:58.577156 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571418 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 10 12:48:58.577156 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571421 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 10 12:48:58.577156 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571423 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 10 12:48:58.577156 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571426 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 10 12:48:58.577156 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571429 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 10 12:48:58.577156 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571431 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 10 12:48:58.577156 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571433 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 10 12:48:58.577156 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571435 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 10 12:48:58.577156 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571438 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 10 12:48:58.577156 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571440 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 10 12:48:58.577156 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571443 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 10 12:48:58.577156 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571445 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 10 12:48:58.577156 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571447 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 10 12:48:58.577156 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571450 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 10 12:48:58.577156 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571452 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 10 12:48:58.577156 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571454 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 10 12:48:58.577657 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571457 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 10 12:48:58.577657 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571460 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 10 12:48:58.577657 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571462 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 10 12:48:58.577657 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571464 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 10 12:48:58.577657 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571467 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 10 12:48:58.577657 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571470 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 10 12:48:58.577657 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:58.571472 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 10 12:48:58.577657 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.571477 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 10 12:48:58.577657 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.571606 2581 server.go:962] "Client rotation is on, will bootstrap in background" Jun 10 12:48:58.577657 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.574269 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 10 12:48:58.577657 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.575664 2581 server.go:1019] "Starting client certificate rotation" Jun 10 12:48:58.577657 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.575757 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 10 12:48:58.577657 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.575793 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 10 12:48:58.578587 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.578573 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 10 12:48:58.588066 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.579690 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 10 12:48:58.591673 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.591660 2581 log.go:25] "Validated CRI v1 runtime API" Jun 10 12:48:58.594857 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.594838 2581 log.go:25] "Validated CRI v1 image API" Jun 10 12:48:58.595683 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.595670 2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 10 12:48:58.597898 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.597882 2581 fs.go:135] Filesystem UUIDs: map[5ea7b87e-8e31-4a5e-982c-7d704a154034:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9f0a28b6-7c1d-4c87-bd66-dabad1cb15aa:/dev/nvme0n1p3] Jun 10 12:48:58.597948 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.597899 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 10 12:48:58.604519 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.604405 2581 manager.go:216] Machine: {Timestamp:2026-06-10 12:48:58.602891664 +0000 UTC m=+0.343106137 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3115553 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26e8af0f13e5d124910dfd35a2f97f SystemUUID:ec26e8af-0f13-e5d1-2491-0dfd35a2f97f BootID:6ad8a8d2-da02-449e-95a4-d69e563b0fb8 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:81:f0:cd:57:d9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:81:f0:cd:57:d9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:36:88:6c:d5:c7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 10 12:48:58.604519 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.604516 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 10 12:48:58.604626 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.604614 2581 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 10 12:48:58.609280 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.609264 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 10 12:48:58.610315 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.610288 2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 10 12:48:58.610462 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.610319 2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-230.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 10 12:48:58.611341 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.611331 2581 topology_manager.go:138] "Creating topology manager with none policy" Jun 10 12:48:58.611382 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.611343 2581 container_manager_linux.go:309] "Creating device plugin manager" Jun 10 12:48:58.611382 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.611350 2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 10 12:48:58.612679 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.612658 2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 10 12:48:58.613477 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.613467 2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 10 12:48:58.616219 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.616208 2581 state_mem.go:36] "Initialized new in-memory state store" Jun 10 12:48:58.616345 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.616335 2581 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 10 12:48:58.616379 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.616364 2581 kubelet.go:486] "Attempting to sync node with API server" Jun 10 12:48:58.616406 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.616380 2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 10 12:48:58.616406 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.616394 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 10 12:48:58.616459 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.616412 2581 kubelet.go:398] "Adding apiserver pod source" Jun 10 12:48:58.616459 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.616435 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 10 12:48:58.617787 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.617538 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 10 12:48:58.624078 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.624051 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 10 12:48:58.627231 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.627214 2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 10 12:48:58.629030 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.629016 2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 10 12:48:58.629095 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.629046 2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 10 12:48:58.629940 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:58.629917 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 10 12:48:58.630004 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:58.629974 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 10 12:48:58.630069 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.630057 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 10 12:48:58.630098 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.630075 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 10 12:48:58.630098 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.630081 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 10 12:48:58.630098 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.630087 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 10 12:48:58.630098 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.630095 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 10 12:48:58.630227 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.630100 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 10 12:48:58.630227 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.630108 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 10 12:48:58.630227 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.630113 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 10 12:48:58.630227 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.630120 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 10 12:48:58.630227 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.630125 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 10 12:48:58.630227 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.630133 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 10 12:48:58.630227 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.630138 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 10 12:48:58.630997 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.630986 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 10 12:48:58.630997 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.630998 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 10 12:48:58.633864 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.633849 2581 server.go:1268] "Started kubelet" Jun 10 12:48:58.633954 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.633926 2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 10 12:48:58.634009 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.633967 2581 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 10 12:48:58.634009 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.633945 2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 10 12:48:58.634864 ip-10-0-128-230 systemd[1]: Started Kubernetes Kubelet. Jun 10 12:48:58.636590 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.636456 2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 10 12:48:58.637374 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.637362 2581 server.go:310] "Adding debug handlers to kubelet server" Jun 10 12:48:58.641226 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.641207 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 10 12:48:58.641328 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.641235 2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 10 12:48:58.641452 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:58.641432 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-230.ec2.internal\" not found" Jun 10 12:48:58.641841 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.641817 2581 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 10 12:48:58.641841 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.641843 2581 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 10 12:48:58.641972 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.641817 2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 10 12:48:58.642021 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.641989 2581 factory.go:55] Registering systemd factory Jun 10 12:48:58.642021 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.642006 2581 factory.go:223] Registration of the systemd container factory successfully Jun 10 12:48:58.642114 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.641990 2581 reconstruct.go:97] "Volume reconstruction finished" Jun 10 12:48:58.642114 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.642068 2581 reconciler.go:29] "Reconciler: start to sync state" Jun 10 12:48:58.642334 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.642289 2581 factory.go:153] Registering CRI-O factory Jun 10 12:48:58.642334 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.642306 2581 factory.go:223] Registration of the crio container factory successfully Jun 10 12:48:58.642676 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.642433 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 10 12:48:58.642676 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.642460 2581 factory.go:103] Registering Raw factory Jun 10 12:48:58.642676 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.642563 2581 manager.go:1196] Started watching for new ooms in manager Jun 10 12:48:58.642866 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:58.642710 2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 10 12:48:58.642866 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.642770 2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-230.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 10 12:48:58.643089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.643075 2581 manager.go:349] Starting recovery of all containers Jun 10 12:48:58.643708 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:58.642642 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-230.ec2.internal.18b7b9233c3b25fb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-230.ec2.internal,UID:ip-10-0-128-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-230.ec2.internal,},FirstTimestamp:2026-06-10 12:48:58.633823739 +0000 UTC m=+0.374038210,LastTimestamp:2026-06-10 12:48:58.633823739 +0000 UTC m=+0.374038210,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-230.ec2.internal,}" Jun 10 12:48:58.649994 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:58.649962 2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 10 12:48:58.650119 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:58.650093 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 10 12:48:58.652912 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.652884 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2dfcw" Jun 10 12:48:58.654016 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.653997 2581 manager.go:354] Recovery completed Jun 10 12:48:58.658608 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.658596 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:48:58.662070 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.662049 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:48:58.662136 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.662076 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:48:58.662136 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.662086 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:48:58.662372 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.662353 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2dfcw" Jun 10 12:48:58.662647 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.662634 2581 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 10 12:48:58.662691 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.662647 2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 10 12:48:58.662691 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.662661 2581 state_mem.go:36] "Initialized new in-memory state store" Jun 10 12:48:58.662691 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.662666 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:48:58.663496 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.663479 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:48:58.663579 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.663512 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:48:58.663579 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.663527 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:48:58.664108 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.664093 2581 policy_none.go:49] "None policy: Start" Jun 10 12:48:58.664199 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.664113 2581 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 10 12:48:58.664199 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.664127 2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 10 12:48:58.664315 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:58.664254 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-230.ec2.internal.18b7b9233dea1923 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-230.ec2.internal,UID:ip-10-0-128-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-230.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-230.ec2.internal,},FirstTimestamp:2026-06-10 12:48:58.662066467 +0000 UTC m=+0.402280939,LastTimestamp:2026-06-10 12:48:58.662066467 +0000 UTC m=+0.402280939,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-230.ec2.internal,}" Jun 10 12:48:58.665132 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.665119 2581 policy_none.go:47] "Start" Jun 10 12:48:58.703775 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.703762 2581 manager.go:337] "Starting Device Plugin manager" Jun 10 12:48:58.711294 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:58.703811 2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 10 12:48:58.711294 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.703823 2581 server.go:87] "Starting device plugin registration server" Jun 10 12:48:58.711294 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.704048 2581 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 10 12:48:58.711294 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.704063 2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 10 12:48:58.711294 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.704135 2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 10 12:48:58.711294 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.704248 2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 10 12:48:58.711294 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.704270 2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 10 12:48:58.711294 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:58.705091 2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 10 12:48:58.711294 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:58.705140 2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-230.ec2.internal\" not found" Jun 10 12:48:58.711294 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.710510 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 10 12:48:58.711818 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.711799 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 10 12:48:58.711887 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.711840 2581 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 10 12:48:58.711887 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.711863 2581 kubelet.go:2448] "Starting kubelet main sync loop" Jun 10 12:48:58.711985 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:58.711934 2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 10 12:48:58.713943 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.713923 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 12:48:58.804319 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.804267 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:48:58.805126 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.805109 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:48:58.805216 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.805144 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:48:58.805216 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.805157 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:48:58.805216 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.805200 2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-230.ec2.internal" Jun 10 12:48:58.812531 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.812511 2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-230.ec2.internal"] Jun 10 12:48:58.812583 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.812576 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:48:58.816220 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.816206 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:48:58.816286 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.816232 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:48:58.816286 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.816245 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:48:58.817627 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.817614 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:48:58.817754 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.817739 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal" Jun 10 12:48:58.817830 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.817811 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:48:58.817974 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.817959 2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-230.ec2.internal" Jun 10 12:48:58.818013 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:58.817984 2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-230.ec2.internal\": node \"ip-10-0-128-230.ec2.internal\" not found" Jun 10 12:48:58.818344 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.818330 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:48:58.818398 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.818352 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:48:58.818398 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.818365 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:48:58.818398 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.818382 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:48:58.818494 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.818403 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:48:58.818494 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.818413 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:48:58.819415 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.819402 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-230.ec2.internal" Jun 10 12:48:58.819461 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.819426 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 10 12:48:58.820115 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.820101 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasSufficientMemory" Jun 10 12:48:58.820211 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.820130 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasNoDiskPressure" Jun 10 12:48:58.820211 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.820144 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeHasSufficientPID" Jun 10 12:48:58.837754 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:58.837731 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-230.ec2.internal\" not found" node="ip-10-0-128-230.ec2.internal" Jun 10 12:48:58.841922 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:58.841907 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-230.ec2.internal\" not found" node="ip-10-0-128-230.ec2.internal" Jun 10 12:48:58.844033 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.844017 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f6d4b102f2deba26bfcb02917e1be38a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal\" (UID: \"f6d4b102f2deba26bfcb02917e1be38a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal" Jun 10 12:48:58.844083 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.844048 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6d4b102f2deba26bfcb02917e1be38a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal\" (UID: \"f6d4b102f2deba26bfcb02917e1be38a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal" Jun 10 12:48:58.844083 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.844064 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d5cbefa49f4f0656a69349468959f722-config\") pod \"kube-apiserver-proxy-ip-10-0-128-230.ec2.internal\" (UID: \"d5cbefa49f4f0656a69349468959f722\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-230.ec2.internal" Jun 10 12:48:58.857141 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:58.857123 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-230.ec2.internal\" not found" Jun 10 12:48:58.944966 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.944943 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f6d4b102f2deba26bfcb02917e1be38a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal\" (UID: \"f6d4b102f2deba26bfcb02917e1be38a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal" Jun 10 12:48:58.945058 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.944970 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6d4b102f2deba26bfcb02917e1be38a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal\" (UID: \"f6d4b102f2deba26bfcb02917e1be38a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal" Jun 10 12:48:58.945058 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.944986 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d5cbefa49f4f0656a69349468959f722-config\") pod \"kube-apiserver-proxy-ip-10-0-128-230.ec2.internal\" (UID: \"d5cbefa49f4f0656a69349468959f722\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-230.ec2.internal" Jun 10 12:48:58.945058 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.945033 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6d4b102f2deba26bfcb02917e1be38a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal\" (UID: \"f6d4b102f2deba26bfcb02917e1be38a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal" Jun 10 12:48:58.945143 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.945053 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d5cbefa49f4f0656a69349468959f722-config\") pod \"kube-apiserver-proxy-ip-10-0-128-230.ec2.internal\" (UID: \"d5cbefa49f4f0656a69349468959f722\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-230.ec2.internal" Jun 10 12:48:58.945143 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:58.945034 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f6d4b102f2deba26bfcb02917e1be38a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal\" (UID: \"f6d4b102f2deba26bfcb02917e1be38a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal" Jun 10 12:48:58.957389 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:58.957369 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-230.ec2.internal\" not found" Jun 10 12:48:59.058280 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:59.058209 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-230.ec2.internal\" not found" Jun 10 12:48:59.141704 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:59.141666 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal" Jun 10 12:48:59.146247 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:59.146229 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-230.ec2.internal" Jun 10 12:48:59.158826 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:59.158807 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-230.ec2.internal\" not found" Jun 10 12:48:59.259337 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:59.259308 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-230.ec2.internal\" not found" Jun 10 12:48:59.359845 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:59.359782 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-230.ec2.internal\" not found" Jun 10 12:48:59.460299 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:59.460270 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-230.ec2.internal\" not found" Jun 10 12:48:59.560769 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:59.560740 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-230.ec2.internal\" not found" Jun 10 12:48:59.574886 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:59.574869 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 10 12:48:59.575010 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:59.574994 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 10 12:48:59.661447 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:48:59.661425 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-230.ec2.internal\" not found" Jun 10 12:48:59.665803 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:59.665776 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-09 12:43:58 +0000 UTC" deadline="2027-11-22 17:37:39.582341323 +0000 UTC" Jun 10 12:48:59.665803 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:59.665800 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12724h48m39.916544025s" Jun 10 12:48:59.721863 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:59.721839 2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 12:48:59.736519 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:59.736350 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 12:48:59.742021 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:59.742003 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal" Jun 10 12:48:59.745364 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:59.745338 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6d4b102f2deba26bfcb02917e1be38a.slice/crio-eca43f7ec75c711c3ac7b5b286215a94a4c736e5a3c31784069dc8075a9cdd3d WatchSource:0}: Error finding container eca43f7ec75c711c3ac7b5b286215a94a4c736e5a3c31784069dc8075a9cdd3d: Status 404 returned error can't find the container with id eca43f7ec75c711c3ac7b5b286215a94a4c736e5a3c31784069dc8075a9cdd3d Jun 10 12:48:59.746928 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:48:59.746900 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5cbefa49f4f0656a69349468959f722.slice/crio-76ad4b01a2a6319ef67d49dc4a53172dc7c0b4e355c83615092163bd34f68f6b WatchSource:0}: Error finding container 76ad4b01a2a6319ef67d49dc4a53172dc7c0b4e355c83615092163bd34f68f6b: Status 404 returned error can't find the container with id 76ad4b01a2a6319ef67d49dc4a53172dc7c0b4e355c83615092163bd34f68f6b Jun 10 12:48:59.751157 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:59.751142 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 12:48:59.757697 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:59.757682 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 10 12:48:59.758841 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:59.758825 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-230.ec2.internal" Jun 10 12:48:59.768250 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:59.768230 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 10 12:48:59.769212 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:48:59.769196 2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 12:49:00.617628 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.617598 2581 apiserver.go:52] "Watching apiserver" Jun 10 12:49:00.630998 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.630966 2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 10 12:49:00.631956 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.631924 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-p427f","openshift-ovn-kubernetes/ovnkube-node-wgg5q","kube-system/konnectivity-agent-g2s2c","kube-system/kube-apiserver-proxy-ip-10-0-128-230.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25","openshift-cluster-node-tuning-operator/tuned-8dqrr","openshift-dns/node-resolver-mtkd7","openshift-image-registry/node-ca-jmkgf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal","openshift-multus/multus-additional-cni-plugins-rzcgc","kube-system/global-pull-secret-syncer-h67vv","openshift-multus/multus-g9cwg","openshift-multus/network-metrics-daemon-2xw2r","openshift-network-diagnostics/network-check-target-vwknl"] Jun 10 12:49:00.633631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.633606 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mtkd7" Jun 10 12:49:00.634933 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.634911 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.636097 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.636077 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:00.636225 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:00.636198 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xw2r" podUID="080cea4a-0afb-41b3-a04c-5763fbb54eac" Jun 10 12:49:00.637280 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.637257 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.638274 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.638244 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 10 12:49:00.638672 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.638650 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jmkgf" Jun 10 12:49:00.639047 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.638944 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 10 12:49:00.640191 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.639853 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.643672 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.641096 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.643672 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.641461 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 10 12:49:00.643672 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.641822 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 10 12:49:00.643672 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.642089 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4ltr9\"" Jun 10 12:49:00.643672 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.642353 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 10 12:49:00.643672 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.642645 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 10 12:49:00.643672 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.642892 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 10 12:49:00.643672 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.643076 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 10 12:49:00.643672 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.643383 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 10 12:49:00.643672 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.643410 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 10 12:49:00.643672 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.643615 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mtc5s\"" Jun 10 12:49:00.643672 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.643670 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 10 12:49:00.644487 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.643737 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 10 12:49:00.644487 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.643918 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 10 12:49:00.644487 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.643926 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lmqtw\"" Jun 10 12:49:00.644487 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.643972 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 10 12:49:00.644487 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.644058 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-p8jtm\"" Jun 10 12:49:00.644724 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.644497 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.644724 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.644532 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 10 12:49:00.644724 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.644716 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 10 12:49:00.644933 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.644914 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 10 12:49:00.645445 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.645406 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 10 12:49:00.645955 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.645940 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nm6sp\"" Jun 10 12:49:00.646404 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.646385 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 10 12:49:00.646497 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.646408 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8jmwx\"" Jun 10 12:49:00.646610 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.646594 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 10 12:49:00.646675 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.646648 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:00.646732 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:00.646718 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h67vv" podUID="907bdd6d-de14-412d-b061-0d62232f8f70" Jun 10 12:49:00.648872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.648847 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p427f" Jun 10 12:49:00.649829 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.649813 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g2s2c" Jun 10 12:49:00.651207 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.651189 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:00.651297 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:00.651255 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwknl" podUID="7ac4149e-b0f6-46fd-bef5-f807a169c5f4" Jun 10 12:49:00.653773 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.653752 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-multus-conf-dir\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.653881 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.653786 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e1fba97-5822-430f-866e-f2155ab419e6-cnibin\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.653881 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.653813 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnwf5\" (UniqueName: \"kubernetes.io/projected/462abeb8-aa7a-475d-9600-1f8f9d579c84-kube-api-access-jnwf5\") pod \"node-resolver-mtkd7\" (UID: \"462abeb8-aa7a-475d-9600-1f8f9d579c84\") " pod="openshift-dns/node-resolver-mtkd7" Jun 10 12:49:00.653881 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.653837 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-run-ovn\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.653881 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.653859 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-cni-bin\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.653881 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.653876 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.654132 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.653915 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8a91de05-d0b9-4192-9583-8de8cf91c175-ovnkube-config\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.654132 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.653959 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-host-run-k8s-cni-cncf-io\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.654132 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.653979 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-host-run-netns\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.654132 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.653994 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e1fba97-5822-430f-866e-f2155ab419e6-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.654132 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654014 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-run-ovn-kubernetes\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.654132 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654042 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-multus-cni-dir\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.654132 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654065 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/462abeb8-aa7a-475d-9600-1f8f9d579c84-hosts-file\") pod \"node-resolver-mtkd7\" (UID: \"462abeb8-aa7a-475d-9600-1f8f9d579c84\") " pod="openshift-dns/node-resolver-mtkd7" Jun 10 12:49:00.654132 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654088 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-log-socket\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.654132 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654111 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-cni-netd\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.654587 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654146 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-etc-selinux\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.654587 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654195 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwkkm\" (UniqueName: \"kubernetes.io/projected/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-kube-api-access-mwkkm\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.654587 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654211 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-host-var-lib-kubelet\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.654587 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654224 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-etc-kubernetes\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.654587 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654238 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e1fba97-5822-430f-866e-f2155ab419e6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.654587 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654278 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-run-systemd\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.654587 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654303 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-sysctl-d\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.654587 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654317 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-cnibin\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.654587 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654331 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e1fba97-5822-430f-866e-f2155ab419e6-system-cni-dir\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.654587 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654352 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9e1fba97-5822-430f-866e-f2155ab419e6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.654587 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654371 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szfcv\" (UniqueName: \"kubernetes.io/projected/9e1fba97-5822-430f-866e-f2155ab419e6-kube-api-access-szfcv\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.654587 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654385 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8a91de05-d0b9-4192-9583-8de8cf91c175-ovnkube-script-lib\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.654587 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654409 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-socket-dir\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.654587 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654432 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-sys-fs\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.654587 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654469 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-run-openvswitch\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.654587 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654508 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-kubelet\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.655363 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654531 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-systemd-units\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.655363 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654547 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-modprobe-d\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.655363 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654575 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-system-cni-dir\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.655363 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654594 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de769e15-3af9-4466-8023-7e09fa20df3e-host\") pod \"node-ca-jmkgf\" (UID: \"de769e15-3af9-4466-8023-7e09fa20df3e\") " pod="openshift-image-registry/node-ca-jmkgf" Jun 10 12:49:00.655363 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654607 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-os-release\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.655363 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654630 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/462abeb8-aa7a-475d-9600-1f8f9d579c84-tmp-dir\") pod \"node-resolver-mtkd7\" (UID: \"462abeb8-aa7a-475d-9600-1f8f9d579c84\") " pod="openshift-dns/node-resolver-mtkd7" Jun 10 12:49:00.655363 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654653 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-var-lib-openvswitch\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.655363 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654680 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drfcg\" (UniqueName: \"kubernetes.io/projected/de769e15-3af9-4466-8023-7e09fa20df3e-kube-api-access-drfcg\") pod \"node-ca-jmkgf\" (UID: \"de769e15-3af9-4466-8023-7e09fa20df3e\") " pod="openshift-image-registry/node-ca-jmkgf" Jun 10 12:49:00.655363 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654702 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-var-lib-kubelet\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.655363 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654732 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f824c6b9-4ac0-432c-9c60-3c60f6deead9-tmp\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.655363 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654748 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4d98\" (UniqueName: \"kubernetes.io/projected/8a91de05-d0b9-4192-9583-8de8cf91c175-kube-api-access-x4d98\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.655363 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654764 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-node-log\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.655363 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654777 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-sysconfig\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.655363 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654789 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-kubernetes\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.655363 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654801 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-lib-modules\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.655363 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654822 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-slash\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.655363 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654854 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-etc-openvswitch\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.655858 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654884 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a91de05-d0b9-4192-9583-8de8cf91c175-ovn-node-metrics-cert\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.655858 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654901 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-multus-socket-dir-parent\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.655858 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654917 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhxt8\" (UniqueName: \"kubernetes.io/projected/f824c6b9-4ac0-432c-9c60-3c60f6deead9-kube-api-access-dhxt8\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.655858 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654932 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e1fba97-5822-430f-866e-f2155ab419e6-os-release\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.655858 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654946 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-run-netns\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.655858 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654960 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-sysctl-conf\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.655858 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.654982 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-systemd\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.655858 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.655007 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-sys\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.655858 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.655032 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-host-var-lib-cni-multus\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.655858 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.655052 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ef1e0594-0603-48b1-a734-a0f38471b4e4-multus-daemon-config\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.655858 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.655068 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-host-run-multus-certs\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.655858 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.655095 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/de769e15-3af9-4466-8023-7e09fa20df3e-serviceca\") pod \"node-ca-jmkgf\" (UID: \"de769e15-3af9-4466-8023-7e09fa20df3e\") " pod="openshift-image-registry/node-ca-jmkgf" Jun 10 12:49:00.655858 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.655116 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-hostroot\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.655858 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.655135 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8a91de05-d0b9-4192-9583-8de8cf91c175-env-overrides\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.655858 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.655161 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-host\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.655858 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.655201 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.656405 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.655230 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-registration-dir\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.656405 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.655249 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq7m5\" (UniqueName: \"kubernetes.io/projected/ef1e0594-0603-48b1-a734-a0f38471b4e4-kube-api-access-gq7m5\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.656405 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.655281 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-device-dir\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.656405 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.655307 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9e1fba97-5822-430f-866e-f2155ab419e6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.656405 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.655323 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7zd6\" (UniqueName: \"kubernetes.io/projected/080cea4a-0afb-41b3-a04c-5763fbb54eac-kube-api-access-p7zd6\") pod \"network-metrics-daemon-2xw2r\" (UID: \"080cea4a-0afb-41b3-a04c-5763fbb54eac\") " pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:00.656405 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.655336 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-run\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.656405 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.655364 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-tuned\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.656405 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.655395 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs\") pod \"network-metrics-daemon-2xw2r\" (UID: \"080cea4a-0afb-41b3-a04c-5763fbb54eac\") " pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:00.656405 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.655416 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef1e0594-0603-48b1-a734-a0f38471b4e4-cni-binary-copy\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.656405 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.655452 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-host-var-lib-cni-bin\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.658270 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.658248 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 10 12:49:00.658369 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.658313 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 10 12:49:00.658369 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.658358 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 10 12:49:00.658604 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.658325 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 10 12:49:00.658748 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.658728 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5bkz5\"" Jun 10 12:49:00.658912 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.658890 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 10 12:49:00.658980 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.658949 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7mz7r\"" Jun 10 12:49:00.658980 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.658966 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 10 12:49:00.660326 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.659018 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 10 12:49:00.660717 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.659243 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 10 12:49:00.660717 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.659260 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wdv9g\"" Jun 10 12:49:00.666984 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.666963 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 10 12:49:00.679673 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.679643 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 10 12:49:00.696140 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.696117 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-v9rvv" Jun 10 12:49:00.706828 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.706800 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-v9rvv" Jun 10 12:49:00.716698 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.716651 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal" event={"ID":"f6d4b102f2deba26bfcb02917e1be38a","Type":"ContainerStarted","Data":"eca43f7ec75c711c3ac7b5b286215a94a4c736e5a3c31784069dc8075a9cdd3d"} Jun 10 12:49:00.717836 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.717811 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-230.ec2.internal" event={"ID":"d5cbefa49f4f0656a69349468959f722","Type":"ContainerStarted","Data":"76ad4b01a2a6319ef67d49dc4a53172dc7c0b4e355c83615092163bd34f68f6b"} Jun 10 12:49:00.742987 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.742964 2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 10 12:49:00.756655 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.756630 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/462abeb8-aa7a-475d-9600-1f8f9d579c84-hosts-file\") pod \"node-resolver-mtkd7\" (UID: \"462abeb8-aa7a-475d-9600-1f8f9d579c84\") " pod="openshift-dns/node-resolver-mtkd7" Jun 10 12:49:00.756750 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.756659 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-log-socket\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.756750 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.756676 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-cni-netd\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.756750 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.756702 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-etc-selinux\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.756750 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.756726 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwkkm\" (UniqueName: \"kubernetes.io/projected/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-kube-api-access-mwkkm\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.756913 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.756749 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-log-socket\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.756913 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.756776 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-cni-netd\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.756913 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.756791 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-etc-selinux\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.756913 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.756797 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/462abeb8-aa7a-475d-9600-1f8f9d579c84-hosts-file\") pod \"node-resolver-mtkd7\" (UID: \"462abeb8-aa7a-475d-9600-1f8f9d579c84\") " pod="openshift-dns/node-resolver-mtkd7" Jun 10 12:49:00.756913 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.756820 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-host-var-lib-kubelet\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.756913 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.756841 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-etc-kubernetes\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.756913 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.756873 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-etc-kubernetes\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.756913 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.756876 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-host-var-lib-kubelet\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.756913 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.756911 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e1fba97-5822-430f-866e-f2155ab419e6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.757268 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.756935 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-run-systemd\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.757268 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.756956 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-sysctl-d\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.757268 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757069 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-run-systemd\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.757268 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757089 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-sysctl-d\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.757268 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757104 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e1fba97-5822-430f-866e-f2155ab419e6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.757268 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757146 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-cnibin\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.757268 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757191 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e1fba97-5822-430f-866e-f2155ab419e6-system-cni-dir\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.757268 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757203 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-cnibin\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.757268 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757218 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9e1fba97-5822-430f-866e-f2155ab419e6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.757268 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757242 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e1fba97-5822-430f-866e-f2155ab419e6-system-cni-dir\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.757268 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757246 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szfcv\" (UniqueName: \"kubernetes.io/projected/9e1fba97-5822-430f-866e-f2155ab419e6-kube-api-access-szfcv\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.757703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757279 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/707bfefc-7461-4cf2-8847-cacc35dcadd4-agent-certs\") pod \"konnectivity-agent-g2s2c\" (UID: \"707bfefc-7461-4cf2-8847-cacc35dcadd4\") " pod="kube-system/konnectivity-agent-g2s2c" Jun 10 12:49:00.757703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757296 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8a91de05-d0b9-4192-9583-8de8cf91c175-ovnkube-script-lib\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.757703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757312 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-socket-dir\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.757703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757326 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-sys-fs\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.757703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757340 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-run-openvswitch\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.757703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757356 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-kubelet\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.757703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757378 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-systemd-units\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.757703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757399 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-modprobe-d\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.757703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757422 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-system-cni-dir\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.757703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757442 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret\") pod \"global-pull-secret-syncer-h67vv\" (UID: \"907bdd6d-de14-412d-b061-0d62232f8f70\") " pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:00.757703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757462 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de769e15-3af9-4466-8023-7e09fa20df3e-host\") pod \"node-ca-jmkgf\" (UID: \"de769e15-3af9-4466-8023-7e09fa20df3e\") " pod="openshift-image-registry/node-ca-jmkgf" Jun 10 12:49:00.757703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757481 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-os-release\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.757703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757506 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/462abeb8-aa7a-475d-9600-1f8f9d579c84-tmp-dir\") pod \"node-resolver-mtkd7\" (UID: \"462abeb8-aa7a-475d-9600-1f8f9d579c84\") " pod="openshift-dns/node-resolver-mtkd7" Jun 10 12:49:00.757703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757531 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-var-lib-openvswitch\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.757703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757548 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drfcg\" (UniqueName: \"kubernetes.io/projected/de769e15-3af9-4466-8023-7e09fa20df3e-kube-api-access-drfcg\") pod \"node-ca-jmkgf\" (UID: \"de769e15-3af9-4466-8023-7e09fa20df3e\") " pod="openshift-image-registry/node-ca-jmkgf" Jun 10 12:49:00.757703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757563 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-var-lib-kubelet\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.757703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757576 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f824c6b9-4ac0-432c-9c60-3c60f6deead9-tmp\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.758344 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757596 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4d98\" (UniqueName: \"kubernetes.io/projected/8a91de05-d0b9-4192-9583-8de8cf91c175-kube-api-access-x4d98\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.758344 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757622 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-node-log\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.758344 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757658 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-sysconfig\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.758344 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757681 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-kubernetes\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.758344 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757696 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-lib-modules\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.758344 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757734 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de769e15-3af9-4466-8023-7e09fa20df3e-host\") pod \"node-ca-jmkgf\" (UID: \"de769e15-3af9-4466-8023-7e09fa20df3e\") " pod="openshift-image-registry/node-ca-jmkgf" Jun 10 12:49:00.758344 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757761 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-slash\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.758344 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757778 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-etc-openvswitch\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.758344 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757787 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-lib-modules\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.758344 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757794 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a91de05-d0b9-4192-9583-8de8cf91c175-ovn-node-metrics-cert\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.758344 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757821 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-multus-socket-dir-parent\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.758344 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757847 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/907bdd6d-de14-412d-b061-0d62232f8f70-dbus\") pod \"global-pull-secret-syncer-h67vv\" (UID: \"907bdd6d-de14-412d-b061-0d62232f8f70\") " pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:00.758344 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757863 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhxt8\" (UniqueName: \"kubernetes.io/projected/f824c6b9-4ac0-432c-9c60-3c60f6deead9-kube-api-access-dhxt8\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.758344 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757883 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e1fba97-5822-430f-866e-f2155ab419e6-os-release\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.758344 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757905 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-run-netns\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.758344 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757925 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-sysctl-conf\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.758344 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757939 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-systemd\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.759089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757953 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-sys\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.759089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757972 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9e1fba97-5822-430f-866e-f2155ab419e6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.759089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.757983 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-host-var-lib-cni-multus\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.759089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758005 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ef1e0594-0603-48b1-a734-a0f38471b4e4-multus-daemon-config\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.759089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758030 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-host-run-multus-certs\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.759089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758052 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/de769e15-3af9-4466-8023-7e09fa20df3e-serviceca\") pod \"node-ca-jmkgf\" (UID: \"de769e15-3af9-4466-8023-7e09fa20df3e\") " pod="openshift-image-registry/node-ca-jmkgf" Jun 10 12:49:00.759089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758086 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-hostroot\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.759089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758090 2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 10 12:49:00.759089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758114 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a321e9c7-bff9-4c6f-a984-6debb7edccd6-iptables-alerter-script\") pod \"iptables-alerter-p427f\" (UID: \"a321e9c7-bff9-4c6f-a984-6debb7edccd6\") " pod="openshift-network-operator/iptables-alerter-p427f" Jun 10 12:49:00.759089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758151 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8a91de05-d0b9-4192-9583-8de8cf91c175-env-overrides\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.759089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758180 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-host\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.759089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758195 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.759089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758211 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-registration-dir\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.759089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758236 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq7m5\" (UniqueName: \"kubernetes.io/projected/ef1e0594-0603-48b1-a734-a0f38471b4e4-kube-api-access-gq7m5\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.759089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758258 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpx6c\" (UniqueName: \"kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c\") pod \"network-check-target-vwknl\" (UID: \"7ac4149e-b0f6-46fd-bef5-f807a169c5f4\") " pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:00.759089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758300 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/907bdd6d-de14-412d-b061-0d62232f8f70-kubelet-config\") pod \"global-pull-secret-syncer-h67vv\" (UID: \"907bdd6d-de14-412d-b061-0d62232f8f70\") " pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:00.759089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758329 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-device-dir\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.759964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758358 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9e1fba97-5822-430f-866e-f2155ab419e6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.759964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758378 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a321e9c7-bff9-4c6f-a984-6debb7edccd6-host-slash\") pod \"iptables-alerter-p427f\" (UID: \"a321e9c7-bff9-4c6f-a984-6debb7edccd6\") " pod="openshift-network-operator/iptables-alerter-p427f" Jun 10 12:49:00.759964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758398 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kfp9\" (UniqueName: \"kubernetes.io/projected/a321e9c7-bff9-4c6f-a984-6debb7edccd6-kube-api-access-5kfp9\") pod \"iptables-alerter-p427f\" (UID: \"a321e9c7-bff9-4c6f-a984-6debb7edccd6\") " pod="openshift-network-operator/iptables-alerter-p427f" Jun 10 12:49:00.759964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758419 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/707bfefc-7461-4cf2-8847-cacc35dcadd4-konnectivity-ca\") pod \"konnectivity-agent-g2s2c\" (UID: \"707bfefc-7461-4cf2-8847-cacc35dcadd4\") " pod="kube-system/konnectivity-agent-g2s2c" Jun 10 12:49:00.759964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758442 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7zd6\" (UniqueName: \"kubernetes.io/projected/080cea4a-0afb-41b3-a04c-5763fbb54eac-kube-api-access-p7zd6\") pod \"network-metrics-daemon-2xw2r\" (UID: \"080cea4a-0afb-41b3-a04c-5763fbb54eac\") " pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:00.759964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758470 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-run\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.759964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758493 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-tuned\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.759964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758524 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs\") pod \"network-metrics-daemon-2xw2r\" (UID: \"080cea4a-0afb-41b3-a04c-5763fbb54eac\") " pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:00.759964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758543 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef1e0594-0603-48b1-a734-a0f38471b4e4-cni-binary-copy\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.759964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758560 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-kubelet\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.759964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758567 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-host-var-lib-cni-bin\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.759964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758602 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-host-var-lib-cni-bin\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.759964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758604 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-multus-conf-dir\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.759964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758633 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e1fba97-5822-430f-866e-f2155ab419e6-cnibin\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.759964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758665 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnwf5\" (UniqueName: \"kubernetes.io/projected/462abeb8-aa7a-475d-9600-1f8f9d579c84-kube-api-access-jnwf5\") pod \"node-resolver-mtkd7\" (UID: \"462abeb8-aa7a-475d-9600-1f8f9d579c84\") " pod="openshift-dns/node-resolver-mtkd7" Jun 10 12:49:00.759964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758691 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-run-ovn\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.759964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758715 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-cni-bin\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.760782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758717 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-os-release\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.760782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758763 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-cni-bin\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.760782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758763 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.760782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758798 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8a91de05-d0b9-4192-9583-8de8cf91c175-ovnkube-config\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.760782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758828 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-host-run-k8s-cni-cncf-io\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.760782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758838 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/462abeb8-aa7a-475d-9600-1f8f9d579c84-tmp-dir\") pod \"node-resolver-mtkd7\" (UID: \"462abeb8-aa7a-475d-9600-1f8f9d579c84\") " pod="openshift-dns/node-resolver-mtkd7" Jun 10 12:49:00.760782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758854 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-host-run-netns\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.760782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758884 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e1fba97-5822-430f-866e-f2155ab419e6-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.760782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758925 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-var-lib-openvswitch\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.760782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758938 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-run-ovn-kubernetes\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.760782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.758967 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-multus-cni-dir\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.760782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759064 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-multus-cni-dir\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.760782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759104 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-systemd-units\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.760782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759136 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-var-lib-kubelet\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.760782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759142 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-hostroot\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.760782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759203 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-etc-openvswitch\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.760782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759225 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-host\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.760782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759240 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.761670 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759273 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.761670 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759296 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8a91de05-d0b9-4192-9583-8de8cf91c175-ovnkube-script-lib\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.761670 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759401 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-socket-dir\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.761670 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759456 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-sys-fs\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.761670 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759494 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-run-openvswitch\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.761670 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759532 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-multus-conf-dir\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.761670 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759571 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e1fba97-5822-430f-866e-f2155ab419e6-cnibin\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.761670 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759617 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8a91de05-d0b9-4192-9583-8de8cf91c175-env-overrides\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.761670 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759692 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-registration-dir\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.761670 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759727 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8a91de05-d0b9-4192-9583-8de8cf91c175-ovnkube-config\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.761670 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759786 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-host-run-k8s-cni-cncf-io\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.761670 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759804 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-run-ovn\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.761670 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759831 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-host-run-netns\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.761670 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759897 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-device-dir\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.761670 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.760007 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-run-ovn-kubernetes\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.761670 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.760062 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-multus-socket-dir-parent\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.761670 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.760087 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-run\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.761670 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.760125 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-sysctl-conf\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.762590 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.760292 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e1fba97-5822-430f-866e-f2155ab419e6-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.762590 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.760349 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-sysconfig\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.762590 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.760360 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-kubernetes\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.762590 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.760301 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-node-log\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.762590 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.760416 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-host-run-multus-certs\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.762590 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.760427 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-sys\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.762590 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.760442 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e1fba97-5822-430f-866e-f2155ab419e6-os-release\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.762590 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.760493 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-host-var-lib-cni-multus\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.762590 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.759152 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-slash\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.762590 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.760576 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8a91de05-d0b9-4192-9583-8de8cf91c175-host-run-netns\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.762590 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:00.760631 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:49:00.762590 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:00.760704 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs podName:080cea4a-0afb-41b3-a04c-5763fbb54eac nodeName:}" failed. No retries permitted until 2026-06-10 12:49:01.260684926 +0000 UTC m=+3.000899408 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs") pod "network-metrics-daemon-2xw2r" (UID: "080cea4a-0afb-41b3-a04c-5763fbb54eac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:49:00.762590 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.760700 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-modprobe-d\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.762590 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.760779 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/de769e15-3af9-4466-8023-7e09fa20df3e-serviceca\") pod \"node-ca-jmkgf\" (UID: \"de769e15-3af9-4466-8023-7e09fa20df3e\") " pod="openshift-image-registry/node-ca-jmkgf" Jun 10 12:49:00.762590 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.760783 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef1e0594-0603-48b1-a734-a0f38471b4e4-system-cni-dir\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.762590 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.760835 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-systemd\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.762590 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.760951 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ef1e0594-0603-48b1-a734-a0f38471b4e4-multus-daemon-config\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.762590 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.760987 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef1e0594-0603-48b1-a734-a0f38471b4e4-cni-binary-copy\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.763489 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.761225 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9e1fba97-5822-430f-866e-f2155ab419e6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.763489 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.761509 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a91de05-d0b9-4192-9583-8de8cf91c175-ovn-node-metrics-cert\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.763489 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.762684 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f824c6b9-4ac0-432c-9c60-3c60f6deead9-tmp\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.763489 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.762715 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f824c6b9-4ac0-432c-9c60-3c60f6deead9-etc-tuned\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.768903 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.768870 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szfcv\" (UniqueName: \"kubernetes.io/projected/9e1fba97-5822-430f-866e-f2155ab419e6-kube-api-access-szfcv\") pod \"multus-additional-cni-plugins-rzcgc\" (UID: \"9e1fba97-5822-430f-866e-f2155ab419e6\") " pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:00.769049 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.769034 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwkkm\" (UniqueName: \"kubernetes.io/projected/8d81ddc8-7ee0-40f5-b6be-5112c0cef252-kube-api-access-mwkkm\") pod \"aws-ebs-csi-driver-node-5fn25\" (UID: \"8d81ddc8-7ee0-40f5-b6be-5112c0cef252\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.773240 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.773210 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq7m5\" (UniqueName: \"kubernetes.io/projected/ef1e0594-0603-48b1-a734-a0f38471b4e4-kube-api-access-gq7m5\") pod \"multus-g9cwg\" (UID: \"ef1e0594-0603-48b1-a734-a0f38471b4e4\") " pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.774938 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.774880 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhxt8\" (UniqueName: \"kubernetes.io/projected/f824c6b9-4ac0-432c-9c60-3c60f6deead9-kube-api-access-dhxt8\") pod \"tuned-8dqrr\" (UID: \"f824c6b9-4ac0-432c-9c60-3c60f6deead9\") " pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.775634 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.775608 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnwf5\" (UniqueName: \"kubernetes.io/projected/462abeb8-aa7a-475d-9600-1f8f9d579c84-kube-api-access-jnwf5\") pod \"node-resolver-mtkd7\" (UID: \"462abeb8-aa7a-475d-9600-1f8f9d579c84\") " pod="openshift-dns/node-resolver-mtkd7" Jun 10 12:49:00.777300 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.777278 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7zd6\" (UniqueName: \"kubernetes.io/projected/080cea4a-0afb-41b3-a04c-5763fbb54eac-kube-api-access-p7zd6\") pod \"network-metrics-daemon-2xw2r\" (UID: \"080cea4a-0afb-41b3-a04c-5763fbb54eac\") " pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:00.777719 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.777697 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drfcg\" (UniqueName: \"kubernetes.io/projected/de769e15-3af9-4466-8023-7e09fa20df3e-kube-api-access-drfcg\") pod \"node-ca-jmkgf\" (UID: \"de769e15-3af9-4466-8023-7e09fa20df3e\") " pod="openshift-image-registry/node-ca-jmkgf" Jun 10 12:49:00.781386 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.781362 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4d98\" (UniqueName: \"kubernetes.io/projected/8a91de05-d0b9-4192-9583-8de8cf91c175-kube-api-access-x4d98\") pod \"ovnkube-node-wgg5q\" (UID: \"8a91de05-d0b9-4192-9583-8de8cf91c175\") " pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.859456 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.859423 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/707bfefc-7461-4cf2-8847-cacc35dcadd4-agent-certs\") pod \"konnectivity-agent-g2s2c\" (UID: \"707bfefc-7461-4cf2-8847-cacc35dcadd4\") " pod="kube-system/konnectivity-agent-g2s2c" Jun 10 12:49:00.859649 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.859463 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret\") pod \"global-pull-secret-syncer-h67vv\" (UID: \"907bdd6d-de14-412d-b061-0d62232f8f70\") " pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:00.859649 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.859492 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/907bdd6d-de14-412d-b061-0d62232f8f70-dbus\") pod \"global-pull-secret-syncer-h67vv\" (UID: \"907bdd6d-de14-412d-b061-0d62232f8f70\") " pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:00.859649 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.859525 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a321e9c7-bff9-4c6f-a984-6debb7edccd6-iptables-alerter-script\") pod \"iptables-alerter-p427f\" (UID: \"a321e9c7-bff9-4c6f-a984-6debb7edccd6\") " pod="openshift-network-operator/iptables-alerter-p427f" Jun 10 12:49:00.859649 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.859547 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpx6c\" (UniqueName: \"kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c\") pod \"network-check-target-vwknl\" (UID: \"7ac4149e-b0f6-46fd-bef5-f807a169c5f4\") " pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:00.859649 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.859589 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/907bdd6d-de14-412d-b061-0d62232f8f70-kubelet-config\") pod \"global-pull-secret-syncer-h67vv\" (UID: \"907bdd6d-de14-412d-b061-0d62232f8f70\") " pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:00.859649 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:00.859593 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:49:00.859649 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.859623 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a321e9c7-bff9-4c6f-a984-6debb7edccd6-host-slash\") pod \"iptables-alerter-p427f\" (UID: \"a321e9c7-bff9-4c6f-a984-6debb7edccd6\") " pod="openshift-network-operator/iptables-alerter-p427f" Jun 10 12:49:00.859998 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.859656 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kfp9\" (UniqueName: \"kubernetes.io/projected/a321e9c7-bff9-4c6f-a984-6debb7edccd6-kube-api-access-5kfp9\") pod \"iptables-alerter-p427f\" (UID: \"a321e9c7-bff9-4c6f-a984-6debb7edccd6\") " pod="openshift-network-operator/iptables-alerter-p427f" Jun 10 12:49:00.859998 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:00.859672 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret podName:907bdd6d-de14-412d-b061-0d62232f8f70 nodeName:}" failed. No retries permitted until 2026-06-10 12:49:01.359650262 +0000 UTC m=+3.099864724 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret") pod "global-pull-secret-syncer-h67vv" (UID: "907bdd6d-de14-412d-b061-0d62232f8f70") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:49:00.859998 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.859711 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/707bfefc-7461-4cf2-8847-cacc35dcadd4-konnectivity-ca\") pod \"konnectivity-agent-g2s2c\" (UID: \"707bfefc-7461-4cf2-8847-cacc35dcadd4\") " pod="kube-system/konnectivity-agent-g2s2c" Jun 10 12:49:00.859998 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.859916 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/907bdd6d-de14-412d-b061-0d62232f8f70-kubelet-config\") pod \"global-pull-secret-syncer-h67vv\" (UID: \"907bdd6d-de14-412d-b061-0d62232f8f70\") " pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:00.859998 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.859979 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a321e9c7-bff9-4c6f-a984-6debb7edccd6-host-slash\") pod \"iptables-alerter-p427f\" (UID: \"a321e9c7-bff9-4c6f-a984-6debb7edccd6\") " pod="openshift-network-operator/iptables-alerter-p427f" Jun 10 12:49:00.860268 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.860022 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/907bdd6d-de14-412d-b061-0d62232f8f70-dbus\") pod \"global-pull-secret-syncer-h67vv\" (UID: \"907bdd6d-de14-412d-b061-0d62232f8f70\") " pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:00.860654 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.860624 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a321e9c7-bff9-4c6f-a984-6debb7edccd6-iptables-alerter-script\") pod \"iptables-alerter-p427f\" (UID: \"a321e9c7-bff9-4c6f-a984-6debb7edccd6\") " pod="openshift-network-operator/iptables-alerter-p427f" Jun 10 12:49:00.860759 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.860698 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/707bfefc-7461-4cf2-8847-cacc35dcadd4-konnectivity-ca\") pod \"konnectivity-agent-g2s2c\" (UID: \"707bfefc-7461-4cf2-8847-cacc35dcadd4\") " pod="kube-system/konnectivity-agent-g2s2c" Jun 10 12:49:00.862599 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.862577 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/707bfefc-7461-4cf2-8847-cacc35dcadd4-agent-certs\") pod \"konnectivity-agent-g2s2c\" (UID: \"707bfefc-7461-4cf2-8847-cacc35dcadd4\") " pod="kube-system/konnectivity-agent-g2s2c" Jun 10 12:49:00.867517 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:00.867496 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:49:00.867517 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:00.867519 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:49:00.867705 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:00.867534 2581 projected.go:196] Error preparing data for projected volume kube-api-access-vpx6c for pod openshift-network-diagnostics/network-check-target-vwknl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:49:00.867705 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:00.867594 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c podName:7ac4149e-b0f6-46fd-bef5-f807a169c5f4 nodeName:}" failed. No retries permitted until 2026-06-10 12:49:01.3675806 +0000 UTC m=+3.107795058 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vpx6c" (UniqueName: "kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c") pod "network-check-target-vwknl" (UID: "7ac4149e-b0f6-46fd-bef5-f807a169c5f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:49:00.872756 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.872738 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kfp9\" (UniqueName: \"kubernetes.io/projected/a321e9c7-bff9-4c6f-a984-6debb7edccd6-kube-api-access-5kfp9\") pod \"iptables-alerter-p427f\" (UID: \"a321e9c7-bff9-4c6f-a984-6debb7edccd6\") " pod="openshift-network-operator/iptables-alerter-p427f" Jun 10 12:49:00.949029 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.949000 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mtkd7" Jun 10 12:49:00.956023 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.956001 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:00.966191 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.966149 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g9cwg" Jun 10 12:49:00.978054 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.978028 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jmkgf" Jun 10 12:49:00.986667 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.986644 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" Jun 10 12:49:00.992305 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.992285 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" Jun 10 12:49:00.999901 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:00.999872 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rzcgc" Jun 10 12:49:01.008558 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:01.008538 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p427f" Jun 10 12:49:01.015048 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:01.015026 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g2s2c" Jun 10 12:49:01.262479 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:01.262404 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs\") pod \"network-metrics-daemon-2xw2r\" (UID: \"080cea4a-0afb-41b3-a04c-5763fbb54eac\") " pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:01.262606 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:01.262517 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:49:01.262606 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:01.262570 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs podName:080cea4a-0afb-41b3-a04c-5763fbb54eac nodeName:}" failed. No retries permitted until 2026-06-10 12:49:02.262556605 +0000 UTC m=+4.002771066 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs") pod "network-metrics-daemon-2xw2r" (UID: "080cea4a-0afb-41b3-a04c-5763fbb54eac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:49:01.363152 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:01.363116 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret\") pod \"global-pull-secret-syncer-h67vv\" (UID: \"907bdd6d-de14-412d-b061-0d62232f8f70\") " pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:01.363348 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:01.363270 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:49:01.363410 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:01.363353 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret podName:907bdd6d-de14-412d-b061-0d62232f8f70 nodeName:}" failed. No retries permitted until 2026-06-10 12:49:02.363335543 +0000 UTC m=+4.103550005 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret") pod "global-pull-secret-syncer-h67vv" (UID: "907bdd6d-de14-412d-b061-0d62232f8f70") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:49:01.463894 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:01.463856 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpx6c\" (UniqueName: \"kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c\") pod \"network-check-target-vwknl\" (UID: \"7ac4149e-b0f6-46fd-bef5-f807a169c5f4\") " pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:01.464057 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:01.464021 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:49:01.464057 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:01.464043 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:49:01.464057 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:01.464055 2581 projected.go:196] Error preparing data for projected volume kube-api-access-vpx6c for pod openshift-network-diagnostics/network-check-target-vwknl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:49:01.464183 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:01.464125 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c podName:7ac4149e-b0f6-46fd-bef5-f807a169c5f4 nodeName:}" failed. No retries permitted until 2026-06-10 12:49:02.464105379 +0000 UTC m=+4.204319885 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpx6c" (UniqueName: "kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c") pod "network-check-target-vwknl" (UID: "7ac4149e-b0f6-46fd-bef5-f807a169c5f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:49:01.594943 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:49:01.594824 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod462abeb8_aa7a_475d_9600_1f8f9d579c84.slice/crio-5a15ae0712d53c359ff4fdc38d8f709dc0e6e155e9572f133198490713a21436 WatchSource:0}: Error finding container 5a15ae0712d53c359ff4fdc38d8f709dc0e6e155e9572f133198490713a21436: Status 404 returned error can't find the container with id 5a15ae0712d53c359ff4fdc38d8f709dc0e6e155e9572f133198490713a21436 Jun 10 12:49:01.596580 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:49:01.596414 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf824c6b9_4ac0_432c_9c60_3c60f6deead9.slice/crio-1dbc5f334402c0c295230b6af467e2505e184b8bc10aaa89b6b21ddcb97e627a WatchSource:0}: Error finding container 1dbc5f334402c0c295230b6af467e2505e184b8bc10aaa89b6b21ddcb97e627a: Status 404 returned error can't find the container with id 1dbc5f334402c0c295230b6af467e2505e184b8bc10aaa89b6b21ddcb97e627a Jun 10 12:49:01.597421 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:49:01.597278 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef1e0594_0603_48b1_a734_a0f38471b4e4.slice/crio-1f225a946dbcc15096092c4f73dcc3dd63033e560ad77f9bba4d6f9f03148b71 WatchSource:0}: Error finding container 1f225a946dbcc15096092c4f73dcc3dd63033e560ad77f9bba4d6f9f03148b71: Status 404 returned error can't find the container with id 1f225a946dbcc15096092c4f73dcc3dd63033e560ad77f9bba4d6f9f03148b71 Jun 10 12:49:01.602697 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:49:01.602674 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a91de05_d0b9_4192_9583_8de8cf91c175.slice/crio-b564902ec81de272ae024a99797a2b964a4c90e3d0b253c7885668ef9844262e WatchSource:0}: Error finding container b564902ec81de272ae024a99797a2b964a4c90e3d0b253c7885668ef9844262e: Status 404 returned error can't find the container with id b564902ec81de272ae024a99797a2b964a4c90e3d0b253c7885668ef9844262e Jun 10 12:49:01.603400 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:49:01.603316 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda321e9c7_bff9_4c6f_a984_6debb7edccd6.slice/crio-f19bdcd5a4dca8aed9803adf6ee924131a3b24113008baccc566a05aa81b1c98 WatchSource:0}: Error finding container f19bdcd5a4dca8aed9803adf6ee924131a3b24113008baccc566a05aa81b1c98: Status 404 returned error can't find the container with id f19bdcd5a4dca8aed9803adf6ee924131a3b24113008baccc566a05aa81b1c98 Jun 10 12:49:01.604696 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:49:01.604675 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e1fba97_5822_430f_866e_f2155ab419e6.slice/crio-20c06f48d568ee994a17182b586c5076674fc339dea4bd9403d5b77684d896bd WatchSource:0}: Error finding container 20c06f48d568ee994a17182b586c5076674fc339dea4bd9403d5b77684d896bd: Status 404 returned error can't find the container with id 20c06f48d568ee994a17182b586c5076674fc339dea4bd9403d5b77684d896bd Jun 10 12:49:01.606529 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:49:01.606485 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde769e15_3af9_4466_8023_7e09fa20df3e.slice/crio-12f7866786afd2a5b27c131e2078d04a2af151acc8c768d9be9c2e0c0a64c66c WatchSource:0}: Error finding container 12f7866786afd2a5b27c131e2078d04a2af151acc8c768d9be9c2e0c0a64c66c: Status 404 returned error can't find the container with id 12f7866786afd2a5b27c131e2078d04a2af151acc8c768d9be9c2e0c0a64c66c Jun 10 12:49:01.607229 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:49:01.606832 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod707bfefc_7461_4cf2_8847_cacc35dcadd4.slice/crio-d512274e2e1a936bcb654f7960793cf7ac858b06bc0671202eb53520539a0140 WatchSource:0}: Error finding container d512274e2e1a936bcb654f7960793cf7ac858b06bc0671202eb53520539a0140: Status 404 returned error can't find the container with id d512274e2e1a936bcb654f7960793cf7ac858b06bc0671202eb53520539a0140 Jun 10 12:49:01.607603 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:49:01.607580 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d81ddc8_7ee0_40f5_b6be_5112c0cef252.slice/crio-cf26d930131f54a10942ce89131cb12d4762432f252dcea8268f8defcfedf9b2 WatchSource:0}: Error finding container cf26d930131f54a10942ce89131cb12d4762432f252dcea8268f8defcfedf9b2: Status 404 returned error can't find the container with id cf26d930131f54a10942ce89131cb12d4762432f252dcea8268f8defcfedf9b2 Jun 10 12:49:01.709142 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:01.709114 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-09 12:44:00 +0000 UTC" deadline="2027-11-09 12:50:15.858200385 +0000 UTC" Jun 10 12:49:01.709142 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:01.709141 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12408h1m14.149062891s" Jun 10 12:49:01.712354 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:01.712332 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:01.712448 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:01.712433 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h67vv" podUID="907bdd6d-de14-412d-b061-0d62232f8f70" Jun 10 12:49:01.722061 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:01.722030 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-230.ec2.internal" event={"ID":"d5cbefa49f4f0656a69349468959f722","Type":"ContainerStarted","Data":"adba2961956bbdc892ff8e437ec4693ae7e2a0fa4a6a99300f6bab724121bd6b"} Jun 10 12:49:01.725757 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:01.725729 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jmkgf" event={"ID":"de769e15-3af9-4466-8023-7e09fa20df3e","Type":"ContainerStarted","Data":"12f7866786afd2a5b27c131e2078d04a2af151acc8c768d9be9c2e0c0a64c66c"} Jun 10 12:49:01.729826 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:01.729796 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" event={"ID":"8d81ddc8-7ee0-40f5-b6be-5112c0cef252","Type":"ContainerStarted","Data":"cf26d930131f54a10942ce89131cb12d4762432f252dcea8268f8defcfedf9b2"} Jun 10 12:49:01.733098 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:01.733073 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g2s2c" event={"ID":"707bfefc-7461-4cf2-8847-cacc35dcadd4","Type":"ContainerStarted","Data":"d512274e2e1a936bcb654f7960793cf7ac858b06bc0671202eb53520539a0140"} Jun 10 12:49:01.735667 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:01.735648 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzcgc" event={"ID":"9e1fba97-5822-430f-866e-f2155ab419e6","Type":"ContainerStarted","Data":"20c06f48d568ee994a17182b586c5076674fc339dea4bd9403d5b77684d896bd"} Jun 10 12:49:01.737542 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:01.737515 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p427f" event={"ID":"a321e9c7-bff9-4c6f-a984-6debb7edccd6","Type":"ContainerStarted","Data":"f19bdcd5a4dca8aed9803adf6ee924131a3b24113008baccc566a05aa81b1c98"} Jun 10 12:49:01.738548 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:01.738528 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g9cwg" event={"ID":"ef1e0594-0603-48b1-a734-a0f38471b4e4","Type":"ContainerStarted","Data":"1f225a946dbcc15096092c4f73dcc3dd63033e560ad77f9bba4d6f9f03148b71"} Jun 10 12:49:01.740230 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:01.740207 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" event={"ID":"8a91de05-d0b9-4192-9583-8de8cf91c175","Type":"ContainerStarted","Data":"b564902ec81de272ae024a99797a2b964a4c90e3d0b253c7885668ef9844262e"} Jun 10 12:49:01.740789 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:01.740751 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-230.ec2.internal" podStartSLOduration=2.7407393559999997 podStartE2EDuration="2.740739356s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:49:01.74001648 +0000 UTC m=+3.480230961" watchObservedRunningTime="2026-06-10 12:49:01.740739356 +0000 UTC m=+3.480953828" Jun 10 12:49:01.743425 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:01.743399 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" event={"ID":"f824c6b9-4ac0-432c-9c60-3c60f6deead9","Type":"ContainerStarted","Data":"1dbc5f334402c0c295230b6af467e2505e184b8bc10aaa89b6b21ddcb97e627a"} Jun 10 12:49:01.744403 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:01.744384 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mtkd7" event={"ID":"462abeb8-aa7a-475d-9600-1f8f9d579c84","Type":"ContainerStarted","Data":"5a15ae0712d53c359ff4fdc38d8f709dc0e6e155e9572f133198490713a21436"} Jun 10 12:49:02.269076 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:02.268440 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs\") pod \"network-metrics-daemon-2xw2r\" (UID: \"080cea4a-0afb-41b3-a04c-5763fbb54eac\") " pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:02.269076 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:02.268617 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:49:02.269076 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:02.268681 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs podName:080cea4a-0afb-41b3-a04c-5763fbb54eac nodeName:}" failed. No retries permitted until 2026-06-10 12:49:04.268662733 +0000 UTC m=+6.008877198 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs") pod "network-metrics-daemon-2xw2r" (UID: "080cea4a-0afb-41b3-a04c-5763fbb54eac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:49:02.369923 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:02.369342 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret\") pod \"global-pull-secret-syncer-h67vv\" (UID: \"907bdd6d-de14-412d-b061-0d62232f8f70\") " pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:02.369923 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:02.369530 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:49:02.369923 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:02.369590 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret podName:907bdd6d-de14-412d-b061-0d62232f8f70 nodeName:}" failed. No retries permitted until 2026-06-10 12:49:04.369572869 +0000 UTC m=+6.109787332 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret") pod "global-pull-secret-syncer-h67vv" (UID: "907bdd6d-de14-412d-b061-0d62232f8f70") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:49:02.470516 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:02.469984 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpx6c\" (UniqueName: \"kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c\") pod \"network-check-target-vwknl\" (UID: \"7ac4149e-b0f6-46fd-bef5-f807a169c5f4\") " pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:02.470516 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:02.470184 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:49:02.470516 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:02.470202 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:49:02.470516 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:02.470211 2581 projected.go:196] Error preparing data for projected volume kube-api-access-vpx6c for pod openshift-network-diagnostics/network-check-target-vwknl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:49:02.470516 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:02.470256 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c podName:7ac4149e-b0f6-46fd-bef5-f807a169c5f4 nodeName:}" failed. No retries permitted until 2026-06-10 12:49:04.470241864 +0000 UTC m=+6.210456323 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpx6c" (UniqueName: "kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c") pod "network-check-target-vwknl" (UID: "7ac4149e-b0f6-46fd-bef5-f807a169c5f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:49:02.710434 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:02.710272 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-09 12:44:00 +0000 UTC" deadline="2028-03-25 01:17:13.096374512 +0000 UTC" Jun 10 12:49:02.710434 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:02.710343 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15684h28m10.386036173s" Jun 10 12:49:02.713763 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:02.713285 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:02.713763 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:02.713413 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xw2r" podUID="080cea4a-0afb-41b3-a04c-5763fbb54eac" Jun 10 12:49:02.714223 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:02.714071 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:02.714223 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:02.714182 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwknl" podUID="7ac4149e-b0f6-46fd-bef5-f807a169c5f4" Jun 10 12:49:02.766806 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:02.766458 2581 generic.go:356] "Generic (PLEG): container finished" podID="f6d4b102f2deba26bfcb02917e1be38a" containerID="c83679d400ed96406175512c6ace3aec54fc3febce9890e173665aa5b7c53952" exitCode=0 Jun 10 12:49:02.766806 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:02.766550 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal" event={"ID":"f6d4b102f2deba26bfcb02917e1be38a","Type":"ContainerDied","Data":"c83679d400ed96406175512c6ace3aec54fc3febce9890e173665aa5b7c53952"} Jun 10 12:49:03.713030 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:03.712998 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:03.713512 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:03.713121 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h67vv" podUID="907bdd6d-de14-412d-b061-0d62232f8f70" Jun 10 12:49:03.792981 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:03.792935 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal" event={"ID":"f6d4b102f2deba26bfcb02917e1be38a","Type":"ContainerStarted","Data":"218762ef7b12c121690582cb39a8b0abb1450c6f99d3cafee5bacd98d7222373"} Jun 10 12:49:04.284065 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:04.284026 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs\") pod \"network-metrics-daemon-2xw2r\" (UID: \"080cea4a-0afb-41b3-a04c-5763fbb54eac\") " pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:04.284275 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:04.284233 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:49:04.284341 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:04.284301 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs podName:080cea4a-0afb-41b3-a04c-5763fbb54eac nodeName:}" failed. No retries permitted until 2026-06-10 12:49:08.284279157 +0000 UTC m=+10.024493615 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs") pod "network-metrics-daemon-2xw2r" (UID: "080cea4a-0afb-41b3-a04c-5763fbb54eac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:49:04.384577 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:04.384540 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret\") pod \"global-pull-secret-syncer-h67vv\" (UID: \"907bdd6d-de14-412d-b061-0d62232f8f70\") " pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:04.384754 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:04.384739 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:49:04.384813 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:04.384803 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret podName:907bdd6d-de14-412d-b061-0d62232f8f70 nodeName:}" failed. No retries permitted until 2026-06-10 12:49:08.384786144 +0000 UTC m=+10.125000608 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret") pod "global-pull-secret-syncer-h67vv" (UID: "907bdd6d-de14-412d-b061-0d62232f8f70") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:49:04.485503 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:04.485463 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpx6c\" (UniqueName: \"kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c\") pod \"network-check-target-vwknl\" (UID: \"7ac4149e-b0f6-46fd-bef5-f807a169c5f4\") " pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:04.485719 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:04.485685 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:49:04.485719 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:04.485714 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:49:04.485824 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:04.485727 2581 projected.go:196] Error preparing data for projected volume kube-api-access-vpx6c for pod openshift-network-diagnostics/network-check-target-vwknl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:49:04.485824 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:04.485786 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c podName:7ac4149e-b0f6-46fd-bef5-f807a169c5f4 nodeName:}" failed. No retries permitted until 2026-06-10 12:49:08.485766154 +0000 UTC m=+10.225980631 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpx6c" (UniqueName: "kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c") pod "network-check-target-vwknl" (UID: "7ac4149e-b0f6-46fd-bef5-f807a169c5f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:49:04.713338 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:04.713265 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:04.713689 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:04.713402 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwknl" podUID="7ac4149e-b0f6-46fd-bef5-f807a169c5f4" Jun 10 12:49:04.713689 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:04.713482 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:04.713689 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:04.713655 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xw2r" podUID="080cea4a-0afb-41b3-a04c-5763fbb54eac" Jun 10 12:49:05.712403 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:05.712362 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:05.712569 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:05.712495 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h67vv" podUID="907bdd6d-de14-412d-b061-0d62232f8f70" Jun 10 12:49:06.713216 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:06.713127 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:06.713216 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:06.713148 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:06.713768 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:06.713293 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xw2r" podUID="080cea4a-0afb-41b3-a04c-5763fbb54eac" Jun 10 12:49:06.713768 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:06.713721 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwknl" podUID="7ac4149e-b0f6-46fd-bef5-f807a169c5f4" Jun 10 12:49:07.712889 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:07.712852 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:07.713078 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:07.712966 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h67vv" podUID="907bdd6d-de14-412d-b061-0d62232f8f70" Jun 10 12:49:08.320057 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:08.320023 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs\") pod \"network-metrics-daemon-2xw2r\" (UID: \"080cea4a-0afb-41b3-a04c-5763fbb54eac\") " pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:08.320497 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:08.320206 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:49:08.320497 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:08.320270 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs podName:080cea4a-0afb-41b3-a04c-5763fbb54eac nodeName:}" failed. No retries permitted until 2026-06-10 12:49:16.320251259 +0000 UTC m=+18.060465718 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs") pod "network-metrics-daemon-2xw2r" (UID: "080cea4a-0afb-41b3-a04c-5763fbb54eac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:49:08.420529 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:08.420492 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret\") pod \"global-pull-secret-syncer-h67vv\" (UID: \"907bdd6d-de14-412d-b061-0d62232f8f70\") " pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:08.420690 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:08.420636 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:49:08.420764 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:08.420703 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret podName:907bdd6d-de14-412d-b061-0d62232f8f70 nodeName:}" failed. No retries permitted until 2026-06-10 12:49:16.420675682 +0000 UTC m=+18.160890147 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret") pod "global-pull-secret-syncer-h67vv" (UID: "907bdd6d-de14-412d-b061-0d62232f8f70") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:49:08.521487 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:08.521449 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpx6c\" (UniqueName: \"kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c\") pod \"network-check-target-vwknl\" (UID: \"7ac4149e-b0f6-46fd-bef5-f807a169c5f4\") " pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:08.521679 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:08.521602 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:49:08.521679 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:08.521628 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:49:08.521679 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:08.521640 2581 projected.go:196] Error preparing data for projected volume kube-api-access-vpx6c for pod openshift-network-diagnostics/network-check-target-vwknl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:49:08.521840 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:08.521703 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c podName:7ac4149e-b0f6-46fd-bef5-f807a169c5f4 nodeName:}" failed. No retries permitted until 2026-06-10 12:49:16.521683944 +0000 UTC m=+18.261898428 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpx6c" (UniqueName: "kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c") pod "network-check-target-vwknl" (UID: "7ac4149e-b0f6-46fd-bef5-f807a169c5f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:49:08.713410 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:08.713338 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:08.713552 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:08.713442 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xw2r" podUID="080cea4a-0afb-41b3-a04c-5763fbb54eac" Jun 10 12:49:08.713552 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:08.713506 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:08.713633 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:08.713597 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwknl" podUID="7ac4149e-b0f6-46fd-bef5-f807a169c5f4" Jun 10 12:49:09.713075 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:09.713040 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:09.713542 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:09.713204 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h67vv" podUID="907bdd6d-de14-412d-b061-0d62232f8f70" Jun 10 12:49:10.713341 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:10.713305 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:10.713803 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:10.713453 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xw2r" podUID="080cea4a-0afb-41b3-a04c-5763fbb54eac" Jun 10 12:49:10.713867 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:10.713812 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:10.713921 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:10.713882 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwknl" podUID="7ac4149e-b0f6-46fd-bef5-f807a169c5f4" Jun 10 12:49:11.712843 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:11.712807 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:11.713013 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:11.712924 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h67vv" podUID="907bdd6d-de14-412d-b061-0d62232f8f70" Jun 10 12:49:12.713156 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:12.713073 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:12.713607 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:12.713232 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xw2r" podUID="080cea4a-0afb-41b3-a04c-5763fbb54eac" Jun 10 12:49:12.713607 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:12.713296 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:12.713607 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:12.713400 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwknl" podUID="7ac4149e-b0f6-46fd-bef5-f807a169c5f4" Jun 10 12:49:13.712352 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:13.712321 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:13.712516 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:13.712451 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h67vv" podUID="907bdd6d-de14-412d-b061-0d62232f8f70" Jun 10 12:49:14.712119 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:14.712051 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:14.712119 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:14.712082 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:14.712642 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:14.712230 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xw2r" podUID="080cea4a-0afb-41b3-a04c-5763fbb54eac" Jun 10 12:49:14.712642 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:14.712348 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwknl" podUID="7ac4149e-b0f6-46fd-bef5-f807a169c5f4" Jun 10 12:49:15.712622 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:15.712429 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:15.713038 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:15.712713 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h67vv" podUID="907bdd6d-de14-412d-b061-0d62232f8f70" Jun 10 12:49:16.383116 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:16.383032 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs\") pod \"network-metrics-daemon-2xw2r\" (UID: \"080cea4a-0afb-41b3-a04c-5763fbb54eac\") " pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:16.383263 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:16.383220 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:49:16.383315 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:16.383297 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs podName:080cea4a-0afb-41b3-a04c-5763fbb54eac nodeName:}" failed. No retries permitted until 2026-06-10 12:49:32.38327297 +0000 UTC m=+34.123487465 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs") pod "network-metrics-daemon-2xw2r" (UID: "080cea4a-0afb-41b3-a04c-5763fbb54eac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:49:16.484094 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:16.484056 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret\") pod \"global-pull-secret-syncer-h67vv\" (UID: \"907bdd6d-de14-412d-b061-0d62232f8f70\") " pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:16.484288 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:16.484187 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:49:16.484288 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:16.484256 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret podName:907bdd6d-de14-412d-b061-0d62232f8f70 nodeName:}" failed. No retries permitted until 2026-06-10 12:49:32.484237521 +0000 UTC m=+34.224451982 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret") pod "global-pull-secret-syncer-h67vv" (UID: "907bdd6d-de14-412d-b061-0d62232f8f70") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:49:16.585322 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:16.585286 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpx6c\" (UniqueName: \"kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c\") pod \"network-check-target-vwknl\" (UID: \"7ac4149e-b0f6-46fd-bef5-f807a169c5f4\") " pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:16.585492 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:16.585475 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:49:16.585566 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:16.585499 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:49:16.585566 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:16.585509 2581 projected.go:196] Error preparing data for projected volume kube-api-access-vpx6c for pod openshift-network-diagnostics/network-check-target-vwknl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:49:16.585566 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:16.585564 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c podName:7ac4149e-b0f6-46fd-bef5-f807a169c5f4 nodeName:}" failed. No retries permitted until 2026-06-10 12:49:32.585551082 +0000 UTC m=+34.325765554 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpx6c" (UniqueName: "kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c") pod "network-check-target-vwknl" (UID: "7ac4149e-b0f6-46fd-bef5-f807a169c5f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:49:16.712466 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:16.712384 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:16.712625 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:16.712386 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:16.712625 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:16.712534 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xw2r" podUID="080cea4a-0afb-41b3-a04c-5763fbb54eac" Jun 10 12:49:16.712625 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:16.712588 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwknl" podUID="7ac4149e-b0f6-46fd-bef5-f807a169c5f4" Jun 10 12:49:17.712514 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:17.712469 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:17.712801 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:17.712616 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h67vv" podUID="907bdd6d-de14-412d-b061-0d62232f8f70" Jun 10 12:49:18.713810 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:18.713781 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:18.714304 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:18.713891 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xw2r" podUID="080cea4a-0afb-41b3-a04c-5763fbb54eac" Jun 10 12:49:18.714304 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:18.713943 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:18.714304 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:18.714036 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwknl" podUID="7ac4149e-b0f6-46fd-bef5-f807a169c5f4" Jun 10 12:49:19.712986 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:19.712951 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:19.713152 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:19.713082 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h67vv" podUID="907bdd6d-de14-412d-b061-0d62232f8f70" Jun 10 12:49:20.713559 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:20.713279 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:20.713559 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:20.713296 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:20.713559 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:20.713443 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xw2r" podUID="080cea4a-0afb-41b3-a04c-5763fbb54eac" Jun 10 12:49:20.713559 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:20.713535 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwknl" podUID="7ac4149e-b0f6-46fd-bef5-f807a169c5f4" Jun 10 12:49:20.827779 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:20.827743 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jmkgf" event={"ID":"de769e15-3af9-4466-8023-7e09fa20df3e","Type":"ContainerStarted","Data":"3ba489afebdcae5c0bd77586a0307a3e57cf3533664ee135c18c4d4a600461f8"} Jun 10 12:49:20.829577 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:20.829545 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" event={"ID":"8d81ddc8-7ee0-40f5-b6be-5112c0cef252","Type":"ContainerStarted","Data":"e286ae4c806f47d8dff87a0f2249c7082df3c173417254192473e218f2832593"} Jun 10 12:49:20.831136 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:20.831111 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g2s2c" event={"ID":"707bfefc-7461-4cf2-8847-cacc35dcadd4","Type":"ContainerStarted","Data":"79ab4220185f3fe78c180ac7f504da19f8943bcae24ff67fd66595df75ae0715"} Jun 10 12:49:20.832662 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:20.832635 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzcgc" event={"ID":"9e1fba97-5822-430f-866e-f2155ab419e6","Type":"ContainerStarted","Data":"e2cc486d42b8d9f90dc6b4e6a290711e9a4a2c77ff4fb1b2b53cdae1b5ffcb51"} Jun 10 12:49:20.835612 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:20.835118 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g9cwg" event={"ID":"ef1e0594-0603-48b1-a734-a0f38471b4e4","Type":"ContainerStarted","Data":"9afea3febaa6d28fd0378356b6ad795aa8b2eeaef184a54ff726cf46f11f22f6"} Jun 10 12:49:20.837352 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:20.837323 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" event={"ID":"8a91de05-d0b9-4192-9583-8de8cf91c175","Type":"ContainerStarted","Data":"0d7626e62071a605b4112f2f287f282f014b3399bb54b56d748ed36ab6738409"} Jun 10 12:49:20.838702 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:20.838675 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" event={"ID":"f824c6b9-4ac0-432c-9c60-3c60f6deead9","Type":"ContainerStarted","Data":"1c8aa9a67e893cde6ccd1a88b98ab118b83d7597888aa7480efaa75016892893"} Jun 10 12:49:20.839951 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:20.839932 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mtkd7" event={"ID":"462abeb8-aa7a-475d-9600-1f8f9d579c84","Type":"ContainerStarted","Data":"ba6ae68f765da216bb3ca80b22d85879aa1cf13e5c7f4a710ff150e3a9e4644e"} Jun 10 12:49:20.845412 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:20.845377 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jmkgf" podStartSLOduration=8.346607885 podStartE2EDuration="22.845366881s" totalImagesPullingTime="14.498758996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:49:20.8453265 +0000 UTC m=+22.585540980" watchObservedRunningTime="2026-06-10 12:49:20.845366881 +0000 UTC m=+22.585581362" Jun 10 12:49:20.845527 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:20.845442 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-230.ec2.internal" podStartSLOduration=21.84543901 podStartE2EDuration="21.84543901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:49:03.811807003 +0000 UTC m=+5.552021486" watchObservedRunningTime="2026-06-10 12:49:20.84543901 +0000 UTC m=+22.585653491" Jun 10 12:49:20.890921 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:20.890873 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g2s2c" podStartSLOduration=3.018079683 podStartE2EDuration="21.890855958s" totalImagesPullingTime="18.872776275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:48:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:49:20.890162766 +0000 UTC m=+22.630377249" watchObservedRunningTime="2026-06-10 12:49:20.890855958 +0000 UTC m=+22.631070446" Jun 10 12:49:20.911838 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:20.911777 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mtkd7" podStartSLOduration=4.114787874 podStartE2EDuration="22.911756793s" totalImagesPullingTime="18.796968919s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:49:20.911251206 +0000 UTC m=+22.651465668" watchObservedRunningTime="2026-06-10 12:49:20.911756793 +0000 UTC m=+22.651971276" Jun 10 12:49:20.932253 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:20.932198 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-g9cwg" podStartSLOduration=4.051448057 podStartE2EDuration="22.932180489s" totalImagesPullingTime="18.880732432s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:49:20.93160487 +0000 UTC m=+22.671819351" watchObservedRunningTime="2026-06-10 12:49:20.932180489 +0000 UTC m=+22.672394962" Jun 10 12:49:20.958472 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:20.958424 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8dqrr" podStartSLOduration=4.165821668 podStartE2EDuration="22.958410884s" totalImagesPullingTime="18.792589216s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:49:20.958071076 +0000 UTC m=+22.698285558" watchObservedRunningTime="2026-06-10 12:49:20.958410884 +0000 UTC m=+22.698625365" Jun 10 12:49:21.712986 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:21.712960 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:21.713207 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:21.713063 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h67vv" podUID="907bdd6d-de14-412d-b061-0d62232f8f70" Jun 10 12:49:21.847248 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:21.847055 2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 10 12:49:21.847597 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:21.847426 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" event={"ID":"8d81ddc8-7ee0-40f5-b6be-5112c0cef252","Type":"ContainerStarted","Data":"3dca135032118998045720c2190e1985e4a4b462b43a9820e6a048165f23d0df"} Jun 10 12:49:21.848815 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:21.848792 2581 generic.go:356] "Generic (PLEG): container finished" podID="9e1fba97-5822-430f-866e-f2155ab419e6" containerID="e2cc486d42b8d9f90dc6b4e6a290711e9a4a2c77ff4fb1b2b53cdae1b5ffcb51" exitCode=0 Jun 10 12:49:21.848943 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:21.848871 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzcgc" event={"ID":"9e1fba97-5822-430f-866e-f2155ab419e6","Type":"ContainerDied","Data":"e2cc486d42b8d9f90dc6b4e6a290711e9a4a2c77ff4fb1b2b53cdae1b5ffcb51"} Jun 10 12:49:21.851660 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:21.851642 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgg5q_8a91de05-d0b9-4192-9583-8de8cf91c175/ovn-acl-logging/0.log" Jun 10 12:49:21.851973 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:21.851951 2581 generic.go:356] "Generic (PLEG): container finished" podID="8a91de05-d0b9-4192-9583-8de8cf91c175" containerID="7da6492a9f4af9ca95eb51f040590a17056669c4aab1c5b019c04419605bfabb" exitCode=1 Jun 10 12:49:21.852067 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:21.852052 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" event={"ID":"8a91de05-d0b9-4192-9583-8de8cf91c175","Type":"ContainerDied","Data":"7da6492a9f4af9ca95eb51f040590a17056669c4aab1c5b019c04419605bfabb"} Jun 10 12:49:21.852139 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:21.852124 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" event={"ID":"8a91de05-d0b9-4192-9583-8de8cf91c175","Type":"ContainerStarted","Data":"ad4928c90c22d62ea3f88dc55d278e5db5af53f4ad33ae78b58a613c2db5e62f"} Jun 10 12:49:21.852218 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:21.852192 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" event={"ID":"8a91de05-d0b9-4192-9583-8de8cf91c175","Type":"ContainerStarted","Data":"588d6ff0f210b7ad3437b2f296c01fff5a6c3c762b66c3693208a7229917fb10"} Jun 10 12:49:21.852278 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:21.852243 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" event={"ID":"8a91de05-d0b9-4192-9583-8de8cf91c175","Type":"ContainerStarted","Data":"78dc47bd190ed684b22e65588c72b205f7b7af9625f570f05c240dbd4b430a5d"} Jun 10 12:49:21.852334 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:21.852291 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" event={"ID":"8a91de05-d0b9-4192-9583-8de8cf91c175","Type":"ContainerStarted","Data":"be7fd48a5534b15e27f1d71bd727404a5dfaa60d61372012343abc9dad31f16d"} Jun 10 12:49:22.330325 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:22.330246 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g2s2c" Jun 10 12:49:22.330877 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:22.330858 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g2s2c" Jun 10 12:49:22.712481 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:22.712454 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:22.712602 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:22.712489 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:22.712655 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:22.712611 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xw2r" podUID="080cea4a-0afb-41b3-a04c-5763fbb54eac" Jun 10 12:49:22.712754 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:22.712726 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwknl" podUID="7ac4149e-b0f6-46fd-bef5-f807a169c5f4" Jun 10 12:49:22.721788 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:22.721688 2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-10T12:49:21.847231375Z","UUID":"a827b54a-105c-4e80-a160-dacc2b3f1326","Handler":null,"Name":"","Endpoint":""} Jun 10 12:49:22.723070 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:22.723050 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 10 12:49:22.723156 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:22.723086 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 10 12:49:22.855833 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:22.855795 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" event={"ID":"8d81ddc8-7ee0-40f5-b6be-5112c0cef252","Type":"ContainerStarted","Data":"26ea0e164edd24bcd11ba750dff58f9eb98a75df6226c9a09a6af8591e401de2"} Jun 10 12:49:22.857489 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:22.857461 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p427f" event={"ID":"a321e9c7-bff9-4c6f-a984-6debb7edccd6","Type":"ContainerStarted","Data":"e0bcbc29356074b8d4efd0d2abf21965429d32e07f3f044d8b6be87a3b84d5f1"} Jun 10 12:49:22.857903 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:22.857876 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g2s2c" Jun 10 12:49:22.858186 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:22.858147 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g2s2c" Jun 10 12:49:22.880306 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:22.880257 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5fn25" podStartSLOduration=3.261541474 podStartE2EDuration="23.8802426s" totalImagesPullingTime="20.618701126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:49:22.879400204 +0000 UTC m=+24.619614685" watchObservedRunningTime="2026-06-10 12:49:22.8802426 +0000 UTC m=+24.620457082" Jun 10 12:49:22.898543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:22.898498 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-p427f" podStartSLOduration=5.109656381 podStartE2EDuration="23.898485686s" totalImagesPullingTime="18.788829305s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:49:22.898041008 +0000 UTC m=+24.638255493" watchObservedRunningTime="2026-06-10 12:49:22.898485686 +0000 UTC m=+24.638700166" Jun 10 12:49:23.712291 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:23.712256 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:23.712481 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:23.712403 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h67vv" podUID="907bdd6d-de14-412d-b061-0d62232f8f70" Jun 10 12:49:23.862997 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:23.862966 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgg5q_8a91de05-d0b9-4192-9583-8de8cf91c175/ovn-acl-logging/0.log" Jun 10 12:49:23.863525 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:23.863366 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" event={"ID":"8a91de05-d0b9-4192-9583-8de8cf91c175","Type":"ContainerStarted","Data":"df7c678f39314a46c7c71970165185889a0a3e9e33ba1393ef7f55283bf54845"} Jun 10 12:49:24.712272 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:24.712243 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:24.712272 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:24.712258 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:24.712487 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:24.712359 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwknl" podUID="7ac4149e-b0f6-46fd-bef5-f807a169c5f4" Jun 10 12:49:24.712487 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:24.712461 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xw2r" podUID="080cea4a-0afb-41b3-a04c-5763fbb54eac" Jun 10 12:49:25.712664 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:25.712630 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:25.713140 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:25.712769 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h67vv" podUID="907bdd6d-de14-412d-b061-0d62232f8f70" Jun 10 12:49:26.712290 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:26.712053 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:26.712388 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:26.712069 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:26.712443 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:26.712408 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xw2r" podUID="080cea4a-0afb-41b3-a04c-5763fbb54eac" Jun 10 12:49:26.712491 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:26.712444 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwknl" podUID="7ac4149e-b0f6-46fd-bef5-f807a169c5f4" Jun 10 12:49:26.871813 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:26.871701 2581 generic.go:356] "Generic (PLEG): container finished" podID="9e1fba97-5822-430f-866e-f2155ab419e6" containerID="b4c4bc6245e8fdd53021578e74be9b56a33b994d258ba218bcecebe389258fbb" exitCode=0 Jun 10 12:49:26.871813 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:26.871778 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzcgc" event={"ID":"9e1fba97-5822-430f-866e-f2155ab419e6","Type":"ContainerDied","Data":"b4c4bc6245e8fdd53021578e74be9b56a33b994d258ba218bcecebe389258fbb"} Jun 10 12:49:26.877786 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:26.877767 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgg5q_8a91de05-d0b9-4192-9583-8de8cf91c175/ovn-acl-logging/0.log" Jun 10 12:49:26.878253 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:26.878138 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" event={"ID":"8a91de05-d0b9-4192-9583-8de8cf91c175","Type":"ContainerStarted","Data":"8212d3697b26281b469ce454ddc153a77f2e19409952cc8408fbba22be8c873c"} Jun 10 12:49:26.878428 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:26.878411 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:26.878584 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:26.878560 2581 scope.go:117] "RemoveContainer" containerID="7da6492a9f4af9ca95eb51f040590a17056669c4aab1c5b019c04419605bfabb" Jun 10 12:49:26.894271 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:26.894253 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:27.713021 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:27.712981 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:27.713160 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:27.713107 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h67vv" podUID="907bdd6d-de14-412d-b061-0d62232f8f70" Jun 10 12:49:27.882560 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:27.882475 2581 generic.go:356] "Generic (PLEG): container finished" podID="9e1fba97-5822-430f-866e-f2155ab419e6" containerID="9118ed82b09e3fbe25b70c49d84cda30ba9c7804ec711a53d5e605744108901f" exitCode=0 Jun 10 12:49:27.882902 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:27.882553 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzcgc" event={"ID":"9e1fba97-5822-430f-866e-f2155ab419e6","Type":"ContainerDied","Data":"9118ed82b09e3fbe25b70c49d84cda30ba9c7804ec711a53d5e605744108901f"} Jun 10 12:49:27.886161 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:27.886134 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgg5q_8a91de05-d0b9-4192-9583-8de8cf91c175/ovn-acl-logging/0.log" Jun 10 12:49:27.886512 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:27.886492 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" event={"ID":"8a91de05-d0b9-4192-9583-8de8cf91c175","Type":"ContainerStarted","Data":"0e754c4bee9f7589d268bf64122631f8e2612d4c4c59f54088b3f33a1409178a"} Jun 10 12:49:27.886673 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:27.886662 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:27.886767 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:27.886749 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:27.902074 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:27.902054 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:49:27.939975 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:27.939936 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" podStartSLOduration=10.994274408 podStartE2EDuration="29.939920446s" totalImagesPullingTime="18.945646038s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:49:27.939479142 +0000 UTC m=+29.679693635" watchObservedRunningTime="2026-06-10 12:49:27.939920446 +0000 UTC m=+29.680134929" Jun 10 12:49:28.713015 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:28.712984 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:28.713159 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:28.713092 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xw2r" podUID="080cea4a-0afb-41b3-a04c-5763fbb54eac" Jun 10 12:49:28.713159 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:28.713114 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:28.713285 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:28.713239 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwknl" podUID="7ac4149e-b0f6-46fd-bef5-f807a169c5f4" Jun 10 12:49:28.868073 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:28.867983 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vwknl"] Jun 10 12:49:28.875742 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:28.875716 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2xw2r"] Jun 10 12:49:28.879262 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:28.879221 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h67vv"] Jun 10 12:49:28.879363 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:28.879329 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:28.879447 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:28.879429 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h67vv" podUID="907bdd6d-de14-412d-b061-0d62232f8f70" Jun 10 12:49:28.891088 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:28.891064 2581 generic.go:356] "Generic (PLEG): container finished" podID="9e1fba97-5822-430f-866e-f2155ab419e6" containerID="d1179b18acb91fb1939e98a5a46384a85f8c13c717905595bdf7e71e205dbeb3" exitCode=0 Jun 10 12:49:28.891491 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:28.891158 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzcgc" event={"ID":"9e1fba97-5822-430f-866e-f2155ab419e6","Type":"ContainerDied","Data":"d1179b18acb91fb1939e98a5a46384a85f8c13c717905595bdf7e71e205dbeb3"} Jun 10 12:49:28.891491 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:28.891220 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:28.891491 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:28.891279 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:28.891491 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:28.891396 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xw2r" podUID="080cea4a-0afb-41b3-a04c-5763fbb54eac" Jun 10 12:49:28.891491 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:28.891465 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwknl" podUID="7ac4149e-b0f6-46fd-bef5-f807a169c5f4" Jun 10 12:49:30.713187 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:30.713082 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:30.713187 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:30.713082 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:30.713939 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:30.713265 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:30.713939 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:30.713288 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwknl" podUID="7ac4149e-b0f6-46fd-bef5-f807a169c5f4" Jun 10 12:49:30.713939 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:30.713354 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h67vv" podUID="907bdd6d-de14-412d-b061-0d62232f8f70" Jun 10 12:49:30.713939 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:30.713445 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xw2r" podUID="080cea4a-0afb-41b3-a04c-5763fbb54eac" Jun 10 12:49:32.398498 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:32.398462 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs\") pod \"network-metrics-daemon-2xw2r\" (UID: \"080cea4a-0afb-41b3-a04c-5763fbb54eac\") " pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:32.399105 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:32.398605 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:49:32.399105 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:32.398665 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs podName:080cea4a-0afb-41b3-a04c-5763fbb54eac nodeName:}" failed. No retries permitted until 2026-06-10 12:50:04.39864684 +0000 UTC m=+66.138861299 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs") pod "network-metrics-daemon-2xw2r" (UID: "080cea4a-0afb-41b3-a04c-5763fbb54eac") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 10 12:49:32.499397 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:32.499362 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret\") pod \"global-pull-secret-syncer-h67vv\" (UID: \"907bdd6d-de14-412d-b061-0d62232f8f70\") " pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:32.499571 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:32.499507 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 10 12:49:32.499637 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:32.499574 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret podName:907bdd6d-de14-412d-b061-0d62232f8f70 nodeName:}" failed. No retries permitted until 2026-06-10 12:50:04.49955533 +0000 UTC m=+66.239769803 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret") pod "global-pull-secret-syncer-h67vv" (UID: "907bdd6d-de14-412d-b061-0d62232f8f70") : object "kube-system"/"original-pull-secret" not registered Jun 10 12:49:32.600197 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:32.600098 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpx6c\" (UniqueName: \"kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c\") pod \"network-check-target-vwknl\" (UID: \"7ac4149e-b0f6-46fd-bef5-f807a169c5f4\") " pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:32.600369 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:32.600307 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 10 12:49:32.600369 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:32.600332 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 10 12:49:32.600369 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:32.600342 2581 projected.go:196] Error preparing data for projected volume kube-api-access-vpx6c for pod openshift-network-diagnostics/network-check-target-vwknl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:49:32.600513 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:32.600402 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c podName:7ac4149e-b0f6-46fd-bef5-f807a169c5f4 nodeName:}" failed. No retries permitted until 2026-06-10 12:50:04.600382521 +0000 UTC m=+66.340596996 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vpx6c" (UniqueName: "kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c") pod "network-check-target-vwknl" (UID: "7ac4149e-b0f6-46fd-bef5-f807a169c5f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 10 12:49:32.712923 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:32.712889 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:32.712923 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:32.712918 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:32.713132 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:32.712889 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:32.713132 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:32.713007 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h67vv" podUID="907bdd6d-de14-412d-b061-0d62232f8f70" Jun 10 12:49:32.713132 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:32.713052 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vwknl" podUID="7ac4149e-b0f6-46fd-bef5-f807a169c5f4" Jun 10 12:49:32.713277 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:32.713133 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xw2r" podUID="080cea4a-0afb-41b3-a04c-5763fbb54eac" Jun 10 12:49:33.906203 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:33.905974 2581 generic.go:356] "Generic (PLEG): container finished" podID="9e1fba97-5822-430f-866e-f2155ab419e6" containerID="82240ed1e34c8bb01cc4146f2eeb9cc88412a97845bce8bed296870da4344608" exitCode=0 Jun 10 12:49:33.906203 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:33.906061 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzcgc" event={"ID":"9e1fba97-5822-430f-866e-f2155ab419e6","Type":"ContainerDied","Data":"82240ed1e34c8bb01cc4146f2eeb9cc88412a97845bce8bed296870da4344608"} Jun 10 12:49:34.627239 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.627149 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-230.ec2.internal" event="NodeReady" Jun 10 12:49:34.627404 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.627307 2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 10 12:49:34.666086 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.666057 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-b777cbcc5-vb962"] Jun 10 12:49:34.690242 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.690215 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz"] Jun 10 12:49:34.690368 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.690346 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.693021 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.692996 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4f92t\"" Jun 10 12:49:34.693021 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.693013 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 10 12:49:34.693672 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.693415 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 10 12:49:34.694698 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.694676 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 10 12:49:34.699145 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.699120 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 10 12:49:34.706095 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.706077 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2pwn5"] Jun 10 12:49:34.706216 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.706201 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:49:34.708890 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.708872 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 10 12:49:34.708983 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.708917 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8jmj2\"" Jun 10 12:49:34.709248 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.709232 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 10 12:49:34.724083 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.724066 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz"] Jun 10 12:49:34.724203 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.724092 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b777cbcc5-vb962"] Jun 10 12:49:34.724203 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.724143 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:49:34.724203 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.724181 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2pwn5" Jun 10 12:49:34.724203 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.724184 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:49:34.724410 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.724264 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:49:34.727002 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.726985 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 10 12:49:34.727534 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.727516 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bxb7s"] Jun 10 12:49:34.727869 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.727853 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 10 12:49:34.727958 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.727878 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 10 12:49:34.727958 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.727947 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jmsw2\"" Jun 10 12:49:34.728076 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.727880 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6n4pt\"" Jun 10 12:49:34.728076 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.727966 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-z79dj\"" Jun 10 12:49:34.730440 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.728537 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 10 12:49:34.730440 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.728682 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 10 12:49:34.730440 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.728878 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 10 12:49:34.750011 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.749994 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2pwn5"] Jun 10 12:49:34.750087 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.750024 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bxb7s"] Jun 10 12:49:34.750126 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.750089 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bxb7s" Jun 10 12:49:34.753020 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.753000 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 10 12:49:34.753115 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.753081 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 10 12:49:34.753327 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.753311 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-79v6m\"" Jun 10 12:49:34.753647 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.753633 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 10 12:49:34.814838 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.814802 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:49:34.814996 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.814847 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-4lhwz\" (UID: \"49614bc3-1721-4b77-90b6-8fcfa6db5ae1\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:49:34.814996 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.814897 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5e7080a1-50fb-441f-989b-c9f2e897f48c-ca-trust-extracted\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.814996 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.814921 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-bound-sa-token\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.814996 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.814936 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-tmp-dir\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:49:34.814996 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.814953 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.815196 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.815002 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5e7080a1-50fb-441f-989b-c9f2e897f48c-trusted-ca\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.815196 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.815037 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5e7080a1-50fb-441f-989b-c9f2e897f48c-installation-pull-secrets\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.815196 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.815065 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-config-volume\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:49:34.815196 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.815088 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-4lhwz\" (UID: \"49614bc3-1721-4b77-90b6-8fcfa6db5ae1\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:49:34.815196 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.815107 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5e7080a1-50fb-441f-989b-c9f2e897f48c-image-registry-private-configuration\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.815196 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.815134 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-certificates\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.815196 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.815184 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j56q6\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-kube-api-access-j56q6\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.815438 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.815209 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wk4k\" (UniqueName: \"kubernetes.io/projected/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-kube-api-access-6wk4k\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:49:34.911445 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.911360 2581 generic.go:356] "Generic (PLEG): container finished" podID="9e1fba97-5822-430f-866e-f2155ab419e6" containerID="d46ff2b01ad5ffe3839e45d6e6013ba3e649039d7e1fc802526325633022e7a9" exitCode=0 Jun 10 12:49:34.911445 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.911411 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzcgc" event={"ID":"9e1fba97-5822-430f-866e-f2155ab419e6","Type":"ContainerDied","Data":"d46ff2b01ad5ffe3839e45d6e6013ba3e649039d7e1fc802526325633022e7a9"} Jun 10 12:49:34.915462 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.915442 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-4lhwz\" (UID: \"49614bc3-1721-4b77-90b6-8fcfa6db5ae1\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:49:34.915551 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.915487 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5e7080a1-50fb-441f-989b-c9f2e897f48c-image-registry-private-configuration\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.915551 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.915516 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-certificates\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.915551 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.915547 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j56q6\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-kube-api-access-j56q6\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.915705 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.915570 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wk4k\" (UniqueName: \"kubernetes.io/projected/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-kube-api-access-6wk4k\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:49:34.915759 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.915720 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5jlq\" (UniqueName: \"kubernetes.io/projected/c4e62b9a-b59d-4df1-a3fb-8185eb2a5472-kube-api-access-q5jlq\") pod \"ingress-canary-bxb7s\" (UID: \"c4e62b9a-b59d-4df1-a3fb-8185eb2a5472\") " pod="openshift-ingress-canary/ingress-canary-bxb7s" Jun 10 12:49:34.915819 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.915800 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:49:34.915869 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.915849 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-4lhwz\" (UID: \"49614bc3-1721-4b77-90b6-8fcfa6db5ae1\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:49:34.915949 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:34.915932 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 12:49:34.916035 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.915941 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5e7080a1-50fb-441f-989b-c9f2e897f48c-ca-trust-extracted\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.916035 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:34.915963 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:49:34.916035 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.915979 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-bound-sa-token\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.916035 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:34.915994 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert podName:49614bc3-1721-4b77-90b6-8fcfa6db5ae1 nodeName:}" failed. No retries permitted until 2026-06-10 12:49:35.415977764 +0000 UTC m=+37.156192226 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-4lhwz" (UID: "49614bc3-1721-4b77-90b6-8fcfa6db5ae1") : secret "networking-console-plugin-cert" not found Jun 10 12:49:34.916035 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:34.916026 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls podName:d0c4887d-c2bc-4f1b-8c0c-29674956c41d nodeName:}" failed. No retries permitted until 2026-06-10 12:49:35.416009202 +0000 UTC m=+37.156223664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls") pod "dns-default-2pwn5" (UID: "d0c4887d-c2bc-4f1b-8c0c-29674956c41d") : secret "dns-default-metrics-tls" not found Jun 10 12:49:34.916350 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.916047 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-tmp-dir\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:49:34.916350 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.916077 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.916350 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.916094 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4e62b9a-b59d-4df1-a3fb-8185eb2a5472-cert\") pod \"ingress-canary-bxb7s\" (UID: \"c4e62b9a-b59d-4df1-a3fb-8185eb2a5472\") " pod="openshift-ingress-canary/ingress-canary-bxb7s" Jun 10 12:49:34.916350 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.916132 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5e7080a1-50fb-441f-989b-c9f2e897f48c-trusted-ca\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.916350 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.916159 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5e7080a1-50fb-441f-989b-c9f2e897f48c-installation-pull-secrets\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.916350 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.916204 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-config-volume\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:49:34.916350 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.916222 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-4lhwz\" (UID: \"49614bc3-1721-4b77-90b6-8fcfa6db5ae1\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:49:34.916350 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:34.916320 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 10 12:49:34.916350 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:34.916333 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b777cbcc5-vb962: secret "image-registry-tls" not found Jun 10 12:49:34.916788 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:34.916381 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls podName:5e7080a1-50fb-441f-989b-c9f2e897f48c nodeName:}" failed. No retries permitted until 2026-06-10 12:49:35.416368271 +0000 UTC m=+37.156582729 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls") pod "image-registry-b777cbcc5-vb962" (UID: "5e7080a1-50fb-441f-989b-c9f2e897f48c") : secret "image-registry-tls" not found Jun 10 12:49:34.916788 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.916401 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-tmp-dir\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:49:34.916788 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.916399 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5e7080a1-50fb-441f-989b-c9f2e897f48c-ca-trust-extracted\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.916788 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.916180 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-certificates\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.916788 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.916644 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-config-volume\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:49:34.917187 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.917149 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5e7080a1-50fb-441f-989b-c9f2e897f48c-trusted-ca\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.920033 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.920013 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5e7080a1-50fb-441f-989b-c9f2e897f48c-installation-pull-secrets\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.920121 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.920046 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5e7080a1-50fb-441f-989b-c9f2e897f48c-image-registry-private-configuration\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.925618 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.925597 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wk4k\" (UniqueName: \"kubernetes.io/projected/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-kube-api-access-6wk4k\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:49:34.925970 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.925955 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-bound-sa-token\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:34.926514 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:34.926500 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j56q6\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-kube-api-access-j56q6\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:35.017250 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:35.017223 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4e62b9a-b59d-4df1-a3fb-8185eb2a5472-cert\") pod \"ingress-canary-bxb7s\" (UID: \"c4e62b9a-b59d-4df1-a3fb-8185eb2a5472\") " pod="openshift-ingress-canary/ingress-canary-bxb7s" Jun 10 12:49:35.017340 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:35.017307 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5jlq\" (UniqueName: \"kubernetes.io/projected/c4e62b9a-b59d-4df1-a3fb-8185eb2a5472-kube-api-access-q5jlq\") pod \"ingress-canary-bxb7s\" (UID: \"c4e62b9a-b59d-4df1-a3fb-8185eb2a5472\") " pod="openshift-ingress-canary/ingress-canary-bxb7s" Jun 10 12:49:35.019782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:35.019754 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4e62b9a-b59d-4df1-a3fb-8185eb2a5472-cert\") pod \"ingress-canary-bxb7s\" (UID: \"c4e62b9a-b59d-4df1-a3fb-8185eb2a5472\") " pod="openshift-ingress-canary/ingress-canary-bxb7s" Jun 10 12:49:35.027389 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:35.027367 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5jlq\" (UniqueName: \"kubernetes.io/projected/c4e62b9a-b59d-4df1-a3fb-8185eb2a5472-kube-api-access-q5jlq\") pod \"ingress-canary-bxb7s\" (UID: \"c4e62b9a-b59d-4df1-a3fb-8185eb2a5472\") " pod="openshift-ingress-canary/ingress-canary-bxb7s" Jun 10 12:49:35.058019 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:35.057997 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bxb7s" Jun 10 12:49:35.229744 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:35.229683 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bxb7s"] Jun 10 12:49:35.234979 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:49:35.234949 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4e62b9a_b59d_4df1_a3fb_8185eb2a5472.slice/crio-35bae63449302d2856c005f6c86fb1987477fd381b62cda6f8d8de59008a3f17 WatchSource:0}: Error finding container 35bae63449302d2856c005f6c86fb1987477fd381b62cda6f8d8de59008a3f17: Status 404 returned error can't find the container with id 35bae63449302d2856c005f6c86fb1987477fd381b62cda6f8d8de59008a3f17 Jun 10 12:49:35.421250 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:35.421052 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:49:35.421428 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:35.421269 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-4lhwz\" (UID: \"49614bc3-1721-4b77-90b6-8fcfa6db5ae1\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:49:35.421428 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:35.421302 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:35.421428 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:35.421219 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:49:35.421428 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:35.421373 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls podName:d0c4887d-c2bc-4f1b-8c0c-29674956c41d nodeName:}" failed. No retries permitted until 2026-06-10 12:49:36.421359029 +0000 UTC m=+38.161573489 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls") pod "dns-default-2pwn5" (UID: "d0c4887d-c2bc-4f1b-8c0c-29674956c41d") : secret "dns-default-metrics-tls" not found Jun 10 12:49:35.421428 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:35.421395 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 10 12:49:35.421428 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:35.421397 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 12:49:35.421428 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:35.421406 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b777cbcc5-vb962: secret "image-registry-tls" not found Jun 10 12:49:35.421759 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:35.421446 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls podName:5e7080a1-50fb-441f-989b-c9f2e897f48c nodeName:}" failed. No retries permitted until 2026-06-10 12:49:36.421435704 +0000 UTC m=+38.161650162 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls") pod "image-registry-b777cbcc5-vb962" (UID: "5e7080a1-50fb-441f-989b-c9f2e897f48c") : secret "image-registry-tls" not found Jun 10 12:49:35.421759 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:35.421459 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert podName:49614bc3-1721-4b77-90b6-8fcfa6db5ae1 nodeName:}" failed. No retries permitted until 2026-06-10 12:49:36.421451869 +0000 UTC m=+38.161666327 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-4lhwz" (UID: "49614bc3-1721-4b77-90b6-8fcfa6db5ae1") : secret "networking-console-plugin-cert" not found Jun 10 12:49:35.918302 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:35.918266 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzcgc" event={"ID":"9e1fba97-5822-430f-866e-f2155ab419e6","Type":"ContainerStarted","Data":"7077f0d9c4cba980d81d1924efb22d2b6be3c0a600011332f88abbab81a06fef"} Jun 10 12:49:35.919403 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:35.919378 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bxb7s" event={"ID":"c4e62b9a-b59d-4df1-a3fb-8185eb2a5472","Type":"ContainerStarted","Data":"35bae63449302d2856c005f6c86fb1987477fd381b62cda6f8d8de59008a3f17"} Jun 10 12:49:35.945352 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:35.945300 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rzcgc" podStartSLOduration=7.571903418 podStartE2EDuration="36.945285665s" totalImagesPullingTime="29.373382247s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:59 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:49:35.945155765 +0000 UTC m=+37.685370245" watchObservedRunningTime="2026-06-10 12:49:35.945285665 +0000 UTC m=+37.685500147" Jun 10 12:49:36.208107 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:36.208029 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:36.299566 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:36.299533 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:36.428981 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:36.428950 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:49:36.428981 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:36.428992 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-4lhwz\" (UID: \"49614bc3-1721-4b77-90b6-8fcfa6db5ae1\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:49:36.429263 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:36.429032 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:36.429263 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:36.429109 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:49:36.429263 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:36.429146 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 10 12:49:36.429263 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:36.429148 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 12:49:36.429263 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:36.429160 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b777cbcc5-vb962: secret "image-registry-tls" not found Jun 10 12:49:36.429263 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:36.429197 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls podName:d0c4887d-c2bc-4f1b-8c0c-29674956c41d nodeName:}" failed. No retries permitted until 2026-06-10 12:49:38.429161089 +0000 UTC m=+40.169375562 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls") pod "dns-default-2pwn5" (UID: "d0c4887d-c2bc-4f1b-8c0c-29674956c41d") : secret "dns-default-metrics-tls" not found Jun 10 12:49:36.429263 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:36.429223 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert podName:49614bc3-1721-4b77-90b6-8fcfa6db5ae1 nodeName:}" failed. No retries permitted until 2026-06-10 12:49:38.429207342 +0000 UTC m=+40.169421800 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-4lhwz" (UID: "49614bc3-1721-4b77-90b6-8fcfa6db5ae1") : secret "networking-console-plugin-cert" not found Jun 10 12:49:36.429263 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:36.429237 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls podName:5e7080a1-50fb-441f-989b-c9f2e897f48c nodeName:}" failed. No retries permitted until 2026-06-10 12:49:38.429230684 +0000 UTC m=+40.169445142 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls") pod "image-registry-b777cbcc5-vb962" (UID: "5e7080a1-50fb-441f-989b-c9f2e897f48c") : secret "image-registry-tls" not found Jun 10 12:49:37.926899 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:37.926856 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bxb7s" event={"ID":"c4e62b9a-b59d-4df1-a3fb-8185eb2a5472","Type":"ContainerStarted","Data":"da1435eb6f3fcfae69b493ebc6d6b2e6d8b8449509e02d0be27991ab6bf9b84f"} Jun 10 12:49:37.957910 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:37.957842 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bxb7s" podStartSLOduration=2.140937105 podStartE2EDuration="3.957826209s" totalImagesPullingTime="1.816889104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:49:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:49:37.957665877 +0000 UTC m=+39.697880358" watchObservedRunningTime="2026-06-10 12:49:37.957826209 +0000 UTC m=+39.698040691" Jun 10 12:49:38.445326 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:38.445286 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-4lhwz\" (UID: \"49614bc3-1721-4b77-90b6-8fcfa6db5ae1\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:49:38.445477 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:38.445351 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:38.445477 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:38.445435 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:49:38.445477 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:38.445447 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 12:49:38.445618 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:38.445511 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert podName:49614bc3-1721-4b77-90b6-8fcfa6db5ae1 nodeName:}" failed. No retries permitted until 2026-06-10 12:49:42.445495583 +0000 UTC m=+44.185710042 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-4lhwz" (UID: "49614bc3-1721-4b77-90b6-8fcfa6db5ae1") : secret "networking-console-plugin-cert" not found Jun 10 12:49:38.445618 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:38.445512 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 10 12:49:38.445618 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:38.445529 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b777cbcc5-vb962: secret "image-registry-tls" not found Jun 10 12:49:38.445618 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:38.445535 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:49:38.445618 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:38.445571 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls podName:5e7080a1-50fb-441f-989b-c9f2e897f48c nodeName:}" failed. No retries permitted until 2026-06-10 12:49:42.445558217 +0000 UTC m=+44.185772675 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls") pod "image-registry-b777cbcc5-vb962" (UID: "5e7080a1-50fb-441f-989b-c9f2e897f48c") : secret "image-registry-tls" not found Jun 10 12:49:38.445618 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:38.445584 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls podName:d0c4887d-c2bc-4f1b-8c0c-29674956c41d nodeName:}" failed. No retries permitted until 2026-06-10 12:49:42.445577926 +0000 UTC m=+44.185792385 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls") pod "dns-default-2pwn5" (UID: "d0c4887d-c2bc-4f1b-8c0c-29674956c41d") : secret "dns-default-metrics-tls" not found Jun 10 12:49:38.761490 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:38.761408 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:38.863907 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:38.863873 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:40.036213 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:40.036156 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:40.136603 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:40.136563 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:40.242485 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:40.242447 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:40.341019 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:40.340920 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:42.472571 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:42.472530 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:49:42.473023 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:42.472581 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-4lhwz\" (UID: \"49614bc3-1721-4b77-90b6-8fcfa6db5ae1\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:49:42.473023 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:42.472626 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:42.473023 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:42.472700 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:49:42.473023 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:42.472720 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 10 12:49:42.473023 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:42.472731 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b777cbcc5-vb962: secret "image-registry-tls" not found Jun 10 12:49:42.473023 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:42.472774 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls podName:5e7080a1-50fb-441f-989b-c9f2e897f48c nodeName:}" failed. No retries permitted until 2026-06-10 12:49:50.472755159 +0000 UTC m=+52.212969618 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls") pod "image-registry-b777cbcc5-vb962" (UID: "5e7080a1-50fb-441f-989b-c9f2e897f48c") : secret "image-registry-tls" not found Jun 10 12:49:42.473023 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:42.472789 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls podName:d0c4887d-c2bc-4f1b-8c0c-29674956c41d nodeName:}" failed. No retries permitted until 2026-06-10 12:49:50.472781277 +0000 UTC m=+52.212995735 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls") pod "dns-default-2pwn5" (UID: "d0c4887d-c2bc-4f1b-8c0c-29674956c41d") : secret "dns-default-metrics-tls" not found Jun 10 12:49:42.473023 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:42.472805 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 12:49:42.473023 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:42.472896 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert podName:49614bc3-1721-4b77-90b6-8fcfa6db5ae1 nodeName:}" failed. No retries permitted until 2026-06-10 12:49:50.472876447 +0000 UTC m=+52.213090918 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-4lhwz" (UID: "49614bc3-1721-4b77-90b6-8fcfa6db5ae1") : secret "networking-console-plugin-cert" not found Jun 10 12:49:42.827347 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:42.827267 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:42.929040 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:42.929001 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:44.336923 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:44.336887 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:44.436699 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:44.436663 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:45.391409 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:45.391364 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:49:50.534981 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:50.534944 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-4lhwz\" (UID: \"49614bc3-1721-4b77-90b6-8fcfa6db5ae1\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:49:50.535585 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:50.534992 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:49:50.535585 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:50.535093 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 10 12:49:50.535585 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:50.535092 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 12:49:50.535585 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:50.535104 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b777cbcc5-vb962: secret "image-registry-tls" not found Jun 10 12:49:50.535585 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:50.535159 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls podName:5e7080a1-50fb-441f-989b-c9f2e897f48c nodeName:}" failed. No retries permitted until 2026-06-10 12:50:06.535144023 +0000 UTC m=+68.275358483 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls") pod "image-registry-b777cbcc5-vb962" (UID: "5e7080a1-50fb-441f-989b-c9f2e897f48c") : secret "image-registry-tls" not found Jun 10 12:49:50.535585 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:50.535199 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert podName:49614bc3-1721-4b77-90b6-8fcfa6db5ae1 nodeName:}" failed. No retries permitted until 2026-06-10 12:50:06.535192381 +0000 UTC m=+68.275406840 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-4lhwz" (UID: "49614bc3-1721-4b77-90b6-8fcfa6db5ae1") : secret "networking-console-plugin-cert" not found Jun 10 12:49:50.535585 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:50.535222 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:49:50.535585 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:50.535294 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:49:50.535585 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:49:50.535320 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls podName:d0c4887d-c2bc-4f1b-8c0c-29674956c41d nodeName:}" failed. No retries permitted until 2026-06-10 12:50:06.53531387 +0000 UTC m=+68.275528328 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls") pod "dns-default-2pwn5" (UID: "d0c4887d-c2bc-4f1b-8c0c-29674956c41d") : secret "dns-default-metrics-tls" not found Jun 10 12:49:59.904499 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:49:59.904473 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wgg5q" Jun 10 12:50:00.070597 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:00.070557 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:50:00.141827 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:00.141796 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:50:03.037881 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:03.037849 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:50:03.111399 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:03.111370 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:50:03.191199 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:03.191158 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:50:04.444384 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:04.444338 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs\") pod \"network-metrics-daemon-2xw2r\" (UID: \"080cea4a-0afb-41b3-a04c-5763fbb54eac\") " pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:50:04.447357 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:04.447336 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 10 12:50:04.454978 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:50:04.454960 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 10 12:50:04.455046 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:50:04.455022 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs podName:080cea4a-0afb-41b3-a04c-5763fbb54eac nodeName:}" failed. No retries permitted until 2026-06-10 12:51:08.455002893 +0000 UTC m=+130.195217352 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs") pod "network-metrics-daemon-2xw2r" (UID: "080cea4a-0afb-41b3-a04c-5763fbb54eac") : secret "metrics-daemon-secret" not found Jun 10 12:50:04.544637 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:04.544606 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret\") pod \"global-pull-secret-syncer-h67vv\" (UID: \"907bdd6d-de14-412d-b061-0d62232f8f70\") " pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:50:04.547852 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:04.547836 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 10 12:50:04.558980 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:04.558949 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/907bdd6d-de14-412d-b061-0d62232f8f70-original-pull-secret\") pod \"global-pull-secret-syncer-h67vv\" (UID: \"907bdd6d-de14-412d-b061-0d62232f8f70\") " pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:50:04.645221 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:04.645191 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpx6c\" (UniqueName: \"kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c\") pod \"network-check-target-vwknl\" (UID: \"7ac4149e-b0f6-46fd-bef5-f807a169c5f4\") " pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:50:04.648243 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:04.648228 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 10 12:50:04.658587 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:04.658572 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 10 12:50:04.668384 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:04.668358 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpx6c\" (UniqueName: \"kubernetes.io/projected/7ac4149e-b0f6-46fd-bef5-f807a169c5f4-kube-api-access-vpx6c\") pod \"network-check-target-vwknl\" (UID: \"7ac4149e-b0f6-46fd-bef5-f807a169c5f4\") " pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:50:04.736460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:04.736408 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h67vv" Jun 10 12:50:04.754619 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:04.754600 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-z79dj\"" Jun 10 12:50:04.762279 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:04.762266 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:50:04.894538 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:04.894510 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h67vv"] Jun 10 12:50:04.897577 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:50:04.897553 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod907bdd6d_de14_412d_b061_0d62232f8f70.slice/crio-97184c11e383c80de36167e6d66edaaf17fffa1be967d41a4b7220a85807fd2f WatchSource:0}: Error finding container 97184c11e383c80de36167e6d66edaaf17fffa1be967d41a4b7220a85807fd2f: Status 404 returned error can't find the container with id 97184c11e383c80de36167e6d66edaaf17fffa1be967d41a4b7220a85807fd2f Jun 10 12:50:04.911437 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:04.911417 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vwknl"] Jun 10 12:50:04.915353 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:50:04.915326 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ac4149e_b0f6_46fd_bef5_f807a169c5f4.slice/crio-b8461a135b4094713ec10d8574e06e2ed6862f2fdf6f38a554f7557f13620a4a WatchSource:0}: Error finding container b8461a135b4094713ec10d8574e06e2ed6862f2fdf6f38a554f7557f13620a4a: Status 404 returned error can't find the container with id b8461a135b4094713ec10d8574e06e2ed6862f2fdf6f38a554f7557f13620a4a Jun 10 12:50:04.987903 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:04.987816 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vwknl" event={"ID":"7ac4149e-b0f6-46fd-bef5-f807a169c5f4","Type":"ContainerStarted","Data":"b8461a135b4094713ec10d8574e06e2ed6862f2fdf6f38a554f7557f13620a4a"} Jun 10 12:50:04.988907 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:04.988867 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h67vv" event={"ID":"907bdd6d-de14-412d-b061-0d62232f8f70","Type":"ContainerStarted","Data":"97184c11e383c80de36167e6d66edaaf17fffa1be967d41a4b7220a85807fd2f"} Jun 10 12:50:06.312017 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:06.311983 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:50:06.422110 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:06.422080 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:50:06.505067 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:06.505001 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:50:06.560853 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:06.560824 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:50:06.561020 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:06.560864 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-4lhwz\" (UID: \"49614bc3-1721-4b77-90b6-8fcfa6db5ae1\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:50:06.561020 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:06.560898 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:50:06.561020 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:50:06.560989 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 12:50:06.561020 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:50:06.560994 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:50:06.561231 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:50:06.561060 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert podName:49614bc3-1721-4b77-90b6-8fcfa6db5ae1 nodeName:}" failed. No retries permitted until 2026-06-10 12:50:38.561043427 +0000 UTC m=+100.301257887 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-4lhwz" (UID: "49614bc3-1721-4b77-90b6-8fcfa6db5ae1") : secret "networking-console-plugin-cert" not found Jun 10 12:50:06.561231 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:50:06.561077 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls podName:d0c4887d-c2bc-4f1b-8c0c-29674956c41d nodeName:}" failed. No retries permitted until 2026-06-10 12:50:38.561069403 +0000 UTC m=+100.301283862 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls") pod "dns-default-2pwn5" (UID: "d0c4887d-c2bc-4f1b-8c0c-29674956c41d") : secret "dns-default-metrics-tls" not found Jun 10 12:50:06.561231 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:50:06.561095 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 10 12:50:06.561231 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:50:06.561111 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b777cbcc5-vb962: secret "image-registry-tls" not found Jun 10 12:50:06.561231 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:50:06.561161 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls podName:5e7080a1-50fb-441f-989b-c9f2e897f48c nodeName:}" failed. No retries permitted until 2026-06-10 12:50:38.561143555 +0000 UTC m=+100.301358031 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls") pod "image-registry-b777cbcc5-vb962" (UID: "5e7080a1-50fb-441f-989b-c9f2e897f48c") : secret "image-registry-tls" not found Jun 10 12:50:10.004545 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:10.004509 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vwknl" event={"ID":"7ac4149e-b0f6-46fd-bef5-f807a169c5f4","Type":"ContainerStarted","Data":"3a37e3e973a5e724a1892bd2a16681d72a0d9c37b18c0ece1c89e90f22227d9f"} Jun 10 12:50:10.004957 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:10.004648 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:50:10.006004 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:10.005983 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h67vv" event={"ID":"907bdd6d-de14-412d-b061-0d62232f8f70","Type":"ContainerStarted","Data":"5f865f10e0808c808dd57b2bfa8987c4f9ba7160004fe186f69e3e09893483b1"} Jun 10 12:50:10.025213 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:10.025139 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vwknl" podStartSLOduration=66.778396144 podStartE2EDuration="1m11.025126713s" totalImagesPullingTime="4.246730569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:48:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:50:10.024449419 +0000 UTC m=+71.764663903" watchObservedRunningTime="2026-06-10 12:50:10.025126713 +0000 UTC m=+71.765341172" Jun 10 12:50:10.043700 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:10.043661 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h67vv" podStartSLOduration=66.773378782 podStartE2EDuration="1m11.043647676s" totalImagesPullingTime="4.270268894s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:48:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:50:10.042616222 +0000 UTC m=+71.782830702" watchObservedRunningTime="2026-06-10 12:50:10.043647676 +0000 UTC m=+71.783862215" Jun 10 12:50:14.925939 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:14.925899 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:50:14.984320 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:14.984283 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:50:36.606343 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:36.606303 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:50:38.588870 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:38.588833 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:50:38.589334 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:38.588879 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-4lhwz\" (UID: \"49614bc3-1721-4b77-90b6-8fcfa6db5ae1\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:50:38.589334 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:50:38.588974 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 12:50:38.589334 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:50:38.588977 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:50:38.589334 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:38.589012 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:50:38.589334 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:50:38.589041 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert podName:49614bc3-1721-4b77-90b6-8fcfa6db5ae1 nodeName:}" failed. No retries permitted until 2026-06-10 12:51:42.58902598 +0000 UTC m=+164.329240439 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-4lhwz" (UID: "49614bc3-1721-4b77-90b6-8fcfa6db5ae1") : secret "networking-console-plugin-cert" not found Jun 10 12:50:38.589334 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:50:38.589073 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls podName:d0c4887d-c2bc-4f1b-8c0c-29674956c41d nodeName:}" failed. No retries permitted until 2026-06-10 12:51:42.589062789 +0000 UTC m=+164.329277249 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls") pod "dns-default-2pwn5" (UID: "d0c4887d-c2bc-4f1b-8c0c-29674956c41d") : secret "dns-default-metrics-tls" not found Jun 10 12:50:38.589334 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:50:38.589099 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 10 12:50:38.589334 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:50:38.589113 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b777cbcc5-vb962: secret "image-registry-tls" not found Jun 10 12:50:38.589334 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:50:38.589205 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls podName:5e7080a1-50fb-441f-989b-c9f2e897f48c nodeName:}" failed. No retries permitted until 2026-06-10 12:51:42.589186409 +0000 UTC m=+164.329400868 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls") pod "image-registry-b777cbcc5-vb962" (UID: "5e7080a1-50fb-441f-989b-c9f2e897f48c") : secret "image-registry-tls" not found Jun 10 12:50:41.010447 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:50:41.010413 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vwknl" Jun 10 12:51:08.496696 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:51:08.496641 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs\") pod \"network-metrics-daemon-2xw2r\" (UID: \"080cea4a-0afb-41b3-a04c-5763fbb54eac\") " pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:51:08.497219 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:51:08.496792 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 10 12:51:08.497219 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:51:08.496868 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs podName:080cea4a-0afb-41b3-a04c-5763fbb54eac nodeName:}" failed. No retries permitted until 2026-06-10 12:53:10.49685386 +0000 UTC m=+252.237068318 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs") pod "network-metrics-daemon-2xw2r" (UID: "080cea4a-0afb-41b3-a04c-5763fbb54eac") : secret "metrics-daemon-secret" not found Jun 10 12:51:37.701117 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:51:37.701057 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-b777cbcc5-vb962" podUID="5e7080a1-50fb-441f-989b-c9f2e897f48c" Jun 10 12:51:37.713415 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:51:37.713382 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" podUID="49614bc3-1721-4b77-90b6-8fcfa6db5ae1" Jun 10 12:51:37.741600 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:51:37.741568 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-2pwn5" podUID="d0c4887d-c2bc-4f1b-8c0c-29674956c41d" Jun 10 12:51:37.746700 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:51:37.746680 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-2xw2r" podUID="080cea4a-0afb-41b3-a04c-5763fbb54eac" Jun 10 12:51:38.216755 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:51:38.216729 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2pwn5" Jun 10 12:51:38.216912 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:51:38.216733 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:51:38.216912 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:51:38.216733 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:51:42.642630 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:51:42.642595 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:51:42.643009 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:51:42.642640 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-4lhwz\" (UID: \"49614bc3-1721-4b77-90b6-8fcfa6db5ae1\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:51:42.643009 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:51:42.642668 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:51:42.643009 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:51:42.642744 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 10 12:51:42.643009 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:51:42.642768 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 12:51:42.643009 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:51:42.642772 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 10 12:51:42.643009 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:51:42.642791 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b777cbcc5-vb962: secret "image-registry-tls" not found Jun 10 12:51:42.643009 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:51:42.642821 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls podName:d0c4887d-c2bc-4f1b-8c0c-29674956c41d nodeName:}" failed. No retries permitted until 2026-06-10 12:53:44.642805742 +0000 UTC m=+286.383020201 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls") pod "dns-default-2pwn5" (UID: "d0c4887d-c2bc-4f1b-8c0c-29674956c41d") : secret "dns-default-metrics-tls" not found Jun 10 12:51:42.643009 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:51:42.642841 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls podName:5e7080a1-50fb-441f-989b-c9f2e897f48c nodeName:}" failed. No retries permitted until 2026-06-10 12:53:44.642827414 +0000 UTC m=+286.383041873 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls") pod "image-registry-b777cbcc5-vb962" (UID: "5e7080a1-50fb-441f-989b-c9f2e897f48c") : secret "image-registry-tls" not found Jun 10 12:51:42.643009 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:51:42.642856 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert podName:49614bc3-1721-4b77-90b6-8fcfa6db5ae1 nodeName:}" failed. No retries permitted until 2026-06-10 12:53:44.642848917 +0000 UTC m=+286.383063377 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-4lhwz" (UID: "49614bc3-1721-4b77-90b6-8fcfa6db5ae1") : secret "networking-console-plugin-cert" not found Jun 10 12:51:49.712619 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:51:49.712526 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:52:34.142134 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:52:34.142100 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:52:53.797879 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:52:53.797844 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:10.547605 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:10.547561 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs\") pod \"network-metrics-daemon-2xw2r\" (UID: \"080cea4a-0afb-41b3-a04c-5763fbb54eac\") " pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:53:10.548077 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:10.547717 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 10 12:53:10.548077 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:10.547799 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs podName:080cea4a-0afb-41b3-a04c-5763fbb54eac nodeName:}" failed. No retries permitted until 2026-06-10 12:55:12.547782466 +0000 UTC m=+374.287996926 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs") pod "network-metrics-daemon-2xw2r" (UID: "080cea4a-0afb-41b3-a04c-5763fbb54eac") : secret "metrics-daemon-secret" not found Jun 10 12:53:21.340727 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:21.340644 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:30.572681 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.572643 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-vw5nk"] Jun 10 12:53:30.574549 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.574528 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-vw5nk" Jun 10 12:53:30.577151 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.577132 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jun 10 12:53:30.577404 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.577388 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jun 10 12:53:30.577505 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.577459 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jun 10 12:53:30.578629 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.578612 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-jwqbd\"" Jun 10 12:53:30.578766 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.578645 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jun 10 12:53:30.579627 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.579608 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2"] Jun 10 12:53:30.581417 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.581340 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6ccc644f47-p5gx4"] Jun 10 12:53:30.581484 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.581457 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2" Jun 10 12:53:30.583147 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.583133 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:30.584071 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.584051 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jun 10 12:53:30.584185 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.584088 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-784nh\"" Jun 10 12:53:30.584185 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.584088 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 10 12:53:30.585361 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.585344 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 10 12:53:30.585445 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.585370 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jun 10 12:53:30.585554 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.585540 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 10 12:53:30.585604 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.585567 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 10 12:53:30.586438 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.586418 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 10 12:53:30.586579 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.586511 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 10 12:53:30.586654 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.586616 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 10 12:53:30.586888 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.586869 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 10 12:53:30.587011 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.586992 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-d2hdb\"" Jun 10 12:53:30.587961 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.587941 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-vw5nk"] Jun 10 12:53:30.597294 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.597265 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6ccc644f47-p5gx4"] Jun 10 12:53:30.602480 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.602462 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2"] Jun 10 12:53:30.677471 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.677438 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-vkhjm"] Jun 10 12:53:30.679380 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.679360 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-vkhjm" Jun 10 12:53:30.684828 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.684802 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jun 10 12:53:30.685289 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.685276 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-xbk67\"" Jun 10 12:53:30.685341 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.685291 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jun 10 12:53:30.687283 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.687259 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-default-certificate\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:30.687399 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.687298 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-stats-auth\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:30.687399 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.687346 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59eb3b55-c728-4a20-bc60-758e5ffc296f-service-ca-bundle\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:30.687399 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.687372 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f57139cf-a599-4dcd-9f59-263cff6bbec2-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-j24l2\" (UID: \"f57139cf-a599-4dcd-9f59-263cff6bbec2\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2" Jun 10 12:53:30.687527 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.687471 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2k8v\" (UniqueName: \"kubernetes.io/projected/59eb3b55-c728-4a20-bc60-758e5ffc296f-kube-api-access-p2k8v\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:30.687527 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.687491 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl9kh\" (UniqueName: \"kubernetes.io/projected/f57139cf-a599-4dcd-9f59-263cff6bbec2-kube-api-access-sl9kh\") pod \"cluster-monitoring-operator-74bbf69bbb-j24l2\" (UID: \"f57139cf-a599-4dcd-9f59-263cff6bbec2\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2" Jun 10 12:53:30.687527 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.687510 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-metrics-certs\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:30.687527 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.687524 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7d867ca-ca36-479d-b023-738ff3414794-config\") pod \"service-ca-operator-6bc5ff68f5-vw5nk\" (UID: \"a7d867ca-ca36-479d-b023-738ff3414794\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-vw5nk" Jun 10 12:53:30.687716 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.687556 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csh84\" (UniqueName: \"kubernetes.io/projected/a7d867ca-ca36-479d-b023-738ff3414794-kube-api-access-csh84\") pod \"service-ca-operator-6bc5ff68f5-vw5nk\" (UID: \"a7d867ca-ca36-479d-b023-738ff3414794\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-vw5nk" Jun 10 12:53:30.687716 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.687584 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7d867ca-ca36-479d-b023-738ff3414794-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-vw5nk\" (UID: \"a7d867ca-ca36-479d-b023-738ff3414794\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-vw5nk" Jun 10 12:53:30.687716 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.687600 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f57139cf-a599-4dcd-9f59-263cff6bbec2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-j24l2\" (UID: \"f57139cf-a599-4dcd-9f59-263cff6bbec2\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2" Jun 10 12:53:30.733853 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.733825 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-vkhjm"] Jun 10 12:53:30.788920 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.788887 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp549\" (UniqueName: \"kubernetes.io/projected/debfe673-395b-4ec5-aa38-bcff94c48874-kube-api-access-xp549\") pod \"volume-data-source-validator-78dd696465-vkhjm\" (UID: \"debfe673-395b-4ec5-aa38-bcff94c48874\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-vkhjm" Jun 10 12:53:30.789062 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.788936 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csh84\" (UniqueName: \"kubernetes.io/projected/a7d867ca-ca36-479d-b023-738ff3414794-kube-api-access-csh84\") pod \"service-ca-operator-6bc5ff68f5-vw5nk\" (UID: \"a7d867ca-ca36-479d-b023-738ff3414794\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-vw5nk" Jun 10 12:53:30.789062 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.788997 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7d867ca-ca36-479d-b023-738ff3414794-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-vw5nk\" (UID: \"a7d867ca-ca36-479d-b023-738ff3414794\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-vw5nk" Jun 10 12:53:30.789062 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.789030 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f57139cf-a599-4dcd-9f59-263cff6bbec2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-j24l2\" (UID: \"f57139cf-a599-4dcd-9f59-263cff6bbec2\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2" Jun 10 12:53:30.789218 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.789080 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-default-certificate\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:30.789218 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.789097 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-stats-auth\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:30.789218 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.789122 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59eb3b55-c728-4a20-bc60-758e5ffc296f-service-ca-bundle\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:30.789218 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.789138 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f57139cf-a599-4dcd-9f59-263cff6bbec2-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-j24l2\" (UID: \"f57139cf-a599-4dcd-9f59-263cff6bbec2\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2" Jun 10 12:53:30.789407 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:30.789222 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 10 12:53:30.789407 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.789261 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2k8v\" (UniqueName: \"kubernetes.io/projected/59eb3b55-c728-4a20-bc60-758e5ffc296f-kube-api-access-p2k8v\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:30.789407 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:30.789306 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f57139cf-a599-4dcd-9f59-263cff6bbec2-cluster-monitoring-operator-tls podName:f57139cf-a599-4dcd-9f59-263cff6bbec2 nodeName:}" failed. No retries permitted until 2026-06-10 12:53:31.289284034 +0000 UTC m=+273.029498492 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f57139cf-a599-4dcd-9f59-263cff6bbec2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-j24l2" (UID: "f57139cf-a599-4dcd-9f59-263cff6bbec2") : secret "cluster-monitoring-operator-tls" not found Jun 10 12:53:30.789407 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.789329 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl9kh\" (UniqueName: \"kubernetes.io/projected/f57139cf-a599-4dcd-9f59-263cff6bbec2-kube-api-access-sl9kh\") pod \"cluster-monitoring-operator-74bbf69bbb-j24l2\" (UID: \"f57139cf-a599-4dcd-9f59-263cff6bbec2\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2" Jun 10 12:53:30.789407 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.789366 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-metrics-certs\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:30.789407 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:30.789399 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59eb3b55-c728-4a20-bc60-758e5ffc296f-service-ca-bundle podName:59eb3b55-c728-4a20-bc60-758e5ffc296f nodeName:}" failed. No retries permitted until 2026-06-10 12:53:31.289382542 +0000 UTC m=+273.029597002 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59eb3b55-c728-4a20-bc60-758e5ffc296f-service-ca-bundle") pod "router-default-6ccc644f47-p5gx4" (UID: "59eb3b55-c728-4a20-bc60-758e5ffc296f") : configmap references non-existent config key: service-ca.crt Jun 10 12:53:30.789713 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.789423 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7d867ca-ca36-479d-b023-738ff3414794-config\") pod \"service-ca-operator-6bc5ff68f5-vw5nk\" (UID: \"a7d867ca-ca36-479d-b023-738ff3414794\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-vw5nk" Jun 10 12:53:30.789713 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:30.789433 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 10 12:53:30.789713 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:30.789472 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-metrics-certs podName:59eb3b55-c728-4a20-bc60-758e5ffc296f nodeName:}" failed. No retries permitted until 2026-06-10 12:53:31.289458093 +0000 UTC m=+273.029672553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-metrics-certs") pod "router-default-6ccc644f47-p5gx4" (UID: "59eb3b55-c728-4a20-bc60-758e5ffc296f") : secret "router-metrics-certs-default" not found Jun 10 12:53:30.789928 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.789902 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f57139cf-a599-4dcd-9f59-263cff6bbec2-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-j24l2\" (UID: \"f57139cf-a599-4dcd-9f59-263cff6bbec2\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2" Jun 10 12:53:30.790189 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.790144 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7d867ca-ca36-479d-b023-738ff3414794-config\") pod \"service-ca-operator-6bc5ff68f5-vw5nk\" (UID: \"a7d867ca-ca36-479d-b023-738ff3414794\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-vw5nk" Jun 10 12:53:30.791509 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.791488 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7d867ca-ca36-479d-b023-738ff3414794-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-vw5nk\" (UID: \"a7d867ca-ca36-479d-b023-738ff3414794\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-vw5nk" Jun 10 12:53:30.791699 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.791685 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-default-certificate\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:30.791749 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.791733 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-stats-auth\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:30.799530 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.799512 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-csh84\" (UniqueName: \"kubernetes.io/projected/a7d867ca-ca36-479d-b023-738ff3414794-kube-api-access-csh84\") pod \"service-ca-operator-6bc5ff68f5-vw5nk\" (UID: \"a7d867ca-ca36-479d-b023-738ff3414794\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-vw5nk" Jun 10 12:53:30.799840 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.799819 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2k8v\" (UniqueName: \"kubernetes.io/projected/59eb3b55-c728-4a20-bc60-758e5ffc296f-kube-api-access-p2k8v\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:30.799882 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.799853 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl9kh\" (UniqueName: \"kubernetes.io/projected/f57139cf-a599-4dcd-9f59-263cff6bbec2-kube-api-access-sl9kh\") pod \"cluster-monitoring-operator-74bbf69bbb-j24l2\" (UID: \"f57139cf-a599-4dcd-9f59-263cff6bbec2\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2" Jun 10 12:53:30.884231 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.884206 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-vw5nk" Jun 10 12:53:30.890774 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.890745 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp549\" (UniqueName: \"kubernetes.io/projected/debfe673-395b-4ec5-aa38-bcff94c48874-kube-api-access-xp549\") pod \"volume-data-source-validator-78dd696465-vkhjm\" (UID: \"debfe673-395b-4ec5-aa38-bcff94c48874\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-vkhjm" Jun 10 12:53:30.901658 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.901630 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:30.903695 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.903673 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp549\" (UniqueName: \"kubernetes.io/projected/debfe673-395b-4ec5-aa38-bcff94c48874-kube-api-access-xp549\") pod \"volume-data-source-validator-78dd696465-vkhjm\" (UID: \"debfe673-395b-4ec5-aa38-bcff94c48874\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-vkhjm" Jun 10 12:53:30.989600 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.989570 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-vkhjm" Jun 10 12:53:31.000086 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:30.999943 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:31.006126 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:31.006096 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-vw5nk"] Jun 10 12:53:31.008662 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:53:31.008642 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7d867ca_ca36_479d_b023_738ff3414794.slice/crio-5ad3b2f8d4f57ec6ef55d37cb5d8bfcf7dfa8dd3642b1bb76c111a41d7a66191 WatchSource:0}: Error finding container 5ad3b2f8d4f57ec6ef55d37cb5d8bfcf7dfa8dd3642b1bb76c111a41d7a66191: Status 404 returned error can't find the container with id 5ad3b2f8d4f57ec6ef55d37cb5d8bfcf7dfa8dd3642b1bb76c111a41d7a66191 Jun 10 12:53:31.120338 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:31.120304 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-vkhjm"] Jun 10 12:53:31.123582 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:53:31.123557 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddebfe673_395b_4ec5_aa38_bcff94c48874.slice/crio-c0d76a957cc982a15c800bfd0ec78270ddd78e836cdff9593ec80a02e03272f2 WatchSource:0}: Error finding container c0d76a957cc982a15c800bfd0ec78270ddd78e836cdff9593ec80a02e03272f2: Status 404 returned error can't find the container with id c0d76a957cc982a15c800bfd0ec78270ddd78e836cdff9593ec80a02e03272f2 Jun 10 12:53:31.295020 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:31.294935 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59eb3b55-c728-4a20-bc60-758e5ffc296f-service-ca-bundle\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:31.295214 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:31.295030 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-metrics-certs\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:31.295214 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:31.295086 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59eb3b55-c728-4a20-bc60-758e5ffc296f-service-ca-bundle podName:59eb3b55-c728-4a20-bc60-758e5ffc296f nodeName:}" failed. No retries permitted until 2026-06-10 12:53:32.295065716 +0000 UTC m=+274.035280176 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59eb3b55-c728-4a20-bc60-758e5ffc296f-service-ca-bundle") pod "router-default-6ccc644f47-p5gx4" (UID: "59eb3b55-c728-4a20-bc60-758e5ffc296f") : configmap references non-existent config key: service-ca.crt Jun 10 12:53:31.295214 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:31.295133 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f57139cf-a599-4dcd-9f59-263cff6bbec2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-j24l2\" (UID: \"f57139cf-a599-4dcd-9f59-263cff6bbec2\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2" Jun 10 12:53:31.295214 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:31.295149 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 10 12:53:31.295393 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:31.295229 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 10 12:53:31.295393 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:31.295231 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-metrics-certs podName:59eb3b55-c728-4a20-bc60-758e5ffc296f nodeName:}" failed. No retries permitted until 2026-06-10 12:53:32.29521491 +0000 UTC m=+274.035429368 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-metrics-certs") pod "router-default-6ccc644f47-p5gx4" (UID: "59eb3b55-c728-4a20-bc60-758e5ffc296f") : secret "router-metrics-certs-default" not found Jun 10 12:53:31.295393 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:31.295268 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f57139cf-a599-4dcd-9f59-263cff6bbec2-cluster-monitoring-operator-tls podName:f57139cf-a599-4dcd-9f59-263cff6bbec2 nodeName:}" failed. No retries permitted until 2026-06-10 12:53:32.295258369 +0000 UTC m=+274.035472827 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f57139cf-a599-4dcd-9f59-263cff6bbec2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-j24l2" (UID: "f57139cf-a599-4dcd-9f59-263cff6bbec2") : secret "cluster-monitoring-operator-tls" not found Jun 10 12:53:31.483315 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:31.483272 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-vkhjm" event={"ID":"debfe673-395b-4ec5-aa38-bcff94c48874","Type":"ContainerStarted","Data":"c0d76a957cc982a15c800bfd0ec78270ddd78e836cdff9593ec80a02e03272f2"} Jun 10 12:53:31.484429 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:31.484396 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-vw5nk" event={"ID":"a7d867ca-ca36-479d-b023-738ff3414794","Type":"ContainerStarted","Data":"5ad3b2f8d4f57ec6ef55d37cb5d8bfcf7dfa8dd3642b1bb76c111a41d7a66191"} Jun 10 12:53:32.304511 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:32.304479 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-metrics-certs\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:32.304877 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:32.304536 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f57139cf-a599-4dcd-9f59-263cff6bbec2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-j24l2\" (UID: \"f57139cf-a599-4dcd-9f59-263cff6bbec2\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2" Jun 10 12:53:32.304877 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:32.304640 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 10 12:53:32.304877 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:32.304650 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 10 12:53:32.304877 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:32.304687 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f57139cf-a599-4dcd-9f59-263cff6bbec2-cluster-monitoring-operator-tls podName:f57139cf-a599-4dcd-9f59-263cff6bbec2 nodeName:}" failed. No retries permitted until 2026-06-10 12:53:34.304674563 +0000 UTC m=+276.044889023 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f57139cf-a599-4dcd-9f59-263cff6bbec2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-j24l2" (UID: "f57139cf-a599-4dcd-9f59-263cff6bbec2") : secret "cluster-monitoring-operator-tls" not found Jun 10 12:53:32.304877 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:32.304712 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-metrics-certs podName:59eb3b55-c728-4a20-bc60-758e5ffc296f nodeName:}" failed. No retries permitted until 2026-06-10 12:53:34.304695258 +0000 UTC m=+276.044909718 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-metrics-certs") pod "router-default-6ccc644f47-p5gx4" (UID: "59eb3b55-c728-4a20-bc60-758e5ffc296f") : secret "router-metrics-certs-default" not found Jun 10 12:53:32.304877 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:32.304742 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59eb3b55-c728-4a20-bc60-758e5ffc296f-service-ca-bundle\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:32.304877 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:32.304868 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59eb3b55-c728-4a20-bc60-758e5ffc296f-service-ca-bundle podName:59eb3b55-c728-4a20-bc60-758e5ffc296f nodeName:}" failed. No retries permitted until 2026-06-10 12:53:34.30485518 +0000 UTC m=+276.045069650 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59eb3b55-c728-4a20-bc60-758e5ffc296f-service-ca-bundle") pod "router-default-6ccc644f47-p5gx4" (UID: "59eb3b55-c728-4a20-bc60-758e5ffc296f") : configmap references non-existent config key: service-ca.crt Jun 10 12:53:32.488283 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:32.488186 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-vkhjm" event={"ID":"debfe673-395b-4ec5-aa38-bcff94c48874","Type":"ContainerStarted","Data":"ed58aa0368221c646fc9b56b516c49764399da9af62b9c113fda19fd3d7c5dab"} Jun 10 12:53:32.507104 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:32.507055 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-vkhjm" podStartSLOduration=1.440070829 podStartE2EDuration="2.507040304s" totalImagesPullingTime="1.066969475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:53:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:53:32.506283974 +0000 UTC m=+274.246498466" watchObservedRunningTime="2026-06-10 12:53:32.507040304 +0000 UTC m=+274.247254784" Jun 10 12:53:32.599653 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:32.599613 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:32.661506 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:32.661453 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:33.492515 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:33.492425 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-vw5nk" event={"ID":"a7d867ca-ca36-479d-b023-738ff3414794","Type":"ContainerStarted","Data":"eb2073e430813db286b3d00070a9265c698cdd2abc48e3ddf8a0a06abaa52297"} Jun 10 12:53:33.512057 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:33.512010 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-vw5nk" podStartSLOduration=1.3686873560000001 podStartE2EDuration="3.511996245s" totalImagesPullingTime="2.143308889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:53:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:53:33.511218017 +0000 UTC m=+275.251432498" watchObservedRunningTime="2026-06-10 12:53:33.511996245 +0000 UTC m=+275.252210765" Jun 10 12:53:34.321730 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:34.321691 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f57139cf-a599-4dcd-9f59-263cff6bbec2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-j24l2\" (UID: \"f57139cf-a599-4dcd-9f59-263cff6bbec2\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2" Jun 10 12:53:34.321924 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:34.321759 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59eb3b55-c728-4a20-bc60-758e5ffc296f-service-ca-bundle\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:34.321924 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:34.321819 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-metrics-certs\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:34.321924 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:34.321845 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 10 12:53:34.321924 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:34.321913 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f57139cf-a599-4dcd-9f59-263cff6bbec2-cluster-monitoring-operator-tls podName:f57139cf-a599-4dcd-9f59-263cff6bbec2 nodeName:}" failed. No retries permitted until 2026-06-10 12:53:38.321895811 +0000 UTC m=+280.062110273 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f57139cf-a599-4dcd-9f59-263cff6bbec2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-j24l2" (UID: "f57139cf-a599-4dcd-9f59-263cff6bbec2") : secret "cluster-monitoring-operator-tls" not found Jun 10 12:53:34.322108 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:34.321930 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59eb3b55-c728-4a20-bc60-758e5ffc296f-service-ca-bundle podName:59eb3b55-c728-4a20-bc60-758e5ffc296f nodeName:}" failed. No retries permitted until 2026-06-10 12:53:38.321921426 +0000 UTC m=+280.062135884 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59eb3b55-c728-4a20-bc60-758e5ffc296f-service-ca-bundle") pod "router-default-6ccc644f47-p5gx4" (UID: "59eb3b55-c728-4a20-bc60-758e5ffc296f") : configmap references non-existent config key: service-ca.crt Jun 10 12:53:34.322108 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:34.321929 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 10 12:53:34.322108 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:34.321990 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-metrics-certs podName:59eb3b55-c728-4a20-bc60-758e5ffc296f nodeName:}" failed. No retries permitted until 2026-06-10 12:53:38.321973594 +0000 UTC m=+280.062188055 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-metrics-certs") pod "router-default-6ccc644f47-p5gx4" (UID: "59eb3b55-c728-4a20-bc60-758e5ffc296f") : secret "router-metrics-certs-default" not found Jun 10 12:53:34.418333 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:34.418303 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:34.478428 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:34.478401 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:34.855846 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:34.855816 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:35.110381 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:35.110344 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:35.211977 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:35.211938 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:35.476328 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:35.476247 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:35.543079 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:35.543046 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:35.869752 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:35.869719 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:35.970414 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:35.970377 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:36.722929 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:36.722896 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-wfvjt"] Jun 10 12:53:36.724948 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:36.724933 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-wfvjt" Jun 10 12:53:36.727692 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:36.727674 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 10 12:53:36.727773 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:36.727681 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 10 12:53:36.728711 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:36.728694 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 10 12:53:36.728711 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:36.728705 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-5jt9r\"" Jun 10 12:53:36.728878 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:36.728751 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 10 12:53:36.734157 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:36.734139 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-wfvjt"] Jun 10 12:53:36.755976 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:36.755947 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:36.842641 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:36.842616 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/48dff5f3-aff8-4bdf-b57b-1c91afc8c09e-signing-key\") pod \"service-ca-7d74d578b8-wfvjt\" (UID: \"48dff5f3-aff8-4bdf-b57b-1c91afc8c09e\") " pod="openshift-service-ca/service-ca-7d74d578b8-wfvjt" Jun 10 12:53:36.842734 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:36.842691 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/48dff5f3-aff8-4bdf-b57b-1c91afc8c09e-signing-cabundle\") pod \"service-ca-7d74d578b8-wfvjt\" (UID: \"48dff5f3-aff8-4bdf-b57b-1c91afc8c09e\") " pod="openshift-service-ca/service-ca-7d74d578b8-wfvjt" Jun 10 12:53:36.842814 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:36.842797 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv46h\" (UniqueName: \"kubernetes.io/projected/48dff5f3-aff8-4bdf-b57b-1c91afc8c09e-kube-api-access-bv46h\") pod \"service-ca-7d74d578b8-wfvjt\" (UID: \"48dff5f3-aff8-4bdf-b57b-1c91afc8c09e\") " pod="openshift-service-ca/service-ca-7d74d578b8-wfvjt" Jun 10 12:53:36.865037 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:36.865011 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:36.943837 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:36.943807 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/48dff5f3-aff8-4bdf-b57b-1c91afc8c09e-signing-key\") pod \"service-ca-7d74d578b8-wfvjt\" (UID: \"48dff5f3-aff8-4bdf-b57b-1c91afc8c09e\") " pod="openshift-service-ca/service-ca-7d74d578b8-wfvjt" Jun 10 12:53:36.944188 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:36.943866 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/48dff5f3-aff8-4bdf-b57b-1c91afc8c09e-signing-cabundle\") pod \"service-ca-7d74d578b8-wfvjt\" (UID: \"48dff5f3-aff8-4bdf-b57b-1c91afc8c09e\") " pod="openshift-service-ca/service-ca-7d74d578b8-wfvjt" Jun 10 12:53:36.944188 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:36.943935 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv46h\" (UniqueName: \"kubernetes.io/projected/48dff5f3-aff8-4bdf-b57b-1c91afc8c09e-kube-api-access-bv46h\") pod \"service-ca-7d74d578b8-wfvjt\" (UID: \"48dff5f3-aff8-4bdf-b57b-1c91afc8c09e\") " pod="openshift-service-ca/service-ca-7d74d578b8-wfvjt" Jun 10 12:53:36.944596 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:36.944571 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/48dff5f3-aff8-4bdf-b57b-1c91afc8c09e-signing-cabundle\") pod \"service-ca-7d74d578b8-wfvjt\" (UID: \"48dff5f3-aff8-4bdf-b57b-1c91afc8c09e\") " pod="openshift-service-ca/service-ca-7d74d578b8-wfvjt" Jun 10 12:53:36.946332 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:36.946312 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/48dff5f3-aff8-4bdf-b57b-1c91afc8c09e-signing-key\") pod \"service-ca-7d74d578b8-wfvjt\" (UID: \"48dff5f3-aff8-4bdf-b57b-1c91afc8c09e\") " pod="openshift-service-ca/service-ca-7d74d578b8-wfvjt" Jun 10 12:53:36.954533 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:36.954515 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv46h\" (UniqueName: \"kubernetes.io/projected/48dff5f3-aff8-4bdf-b57b-1c91afc8c09e-kube-api-access-bv46h\") pod \"service-ca-7d74d578b8-wfvjt\" (UID: \"48dff5f3-aff8-4bdf-b57b-1c91afc8c09e\") " pod="openshift-service-ca/service-ca-7d74d578b8-wfvjt" Jun 10 12:53:36.956710 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:36.956688 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:37.033832 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:37.033766 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-wfvjt" Jun 10 12:53:37.058401 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:37.058367 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:37.157376 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:37.157346 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-wfvjt"] Jun 10 12:53:37.160582 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:53:37.160552 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48dff5f3_aff8_4bdf_b57b_1c91afc8c09e.slice/crio-2d2e543a1b3398157661eb6498489eb9d4b7f5ef4788580a0dc07dea96d75c4a WatchSource:0}: Error finding container 2d2e543a1b3398157661eb6498489eb9d4b7f5ef4788580a0dc07dea96d75c4a: Status 404 returned error can't find the container with id 2d2e543a1b3398157661eb6498489eb9d4b7f5ef4788580a0dc07dea96d75c4a Jun 10 12:53:37.505236 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:37.505197 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-wfvjt" event={"ID":"48dff5f3-aff8-4bdf-b57b-1c91afc8c09e","Type":"ContainerStarted","Data":"7daa43e97b16a73d383e84580f3818bba7f75ab13977a1b39bb3dfb262df9e9d"} Jun 10 12:53:37.505424 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:37.505251 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-wfvjt" event={"ID":"48dff5f3-aff8-4bdf-b57b-1c91afc8c09e","Type":"ContainerStarted","Data":"2d2e543a1b3398157661eb6498489eb9d4b7f5ef4788580a0dc07dea96d75c4a"} Jun 10 12:53:37.525978 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:37.525931 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7d74d578b8-wfvjt" podStartSLOduration=1.5259117039999999 podStartE2EDuration="1.525911704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:53:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:53:37.525297692 +0000 UTC m=+279.265512172" watchObservedRunningTime="2026-06-10 12:53:37.525911704 +0000 UTC m=+279.266126184" Jun 10 12:53:37.595873 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:37.595848 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mtkd7_462abeb8-aa7a-475d-9600-1f8f9d579c84/dns-node-resolver/0.log" Jun 10 12:53:38.103419 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:38.103383 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:38.355087 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:38.354991 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f57139cf-a599-4dcd-9f59-263cff6bbec2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-j24l2\" (UID: \"f57139cf-a599-4dcd-9f59-263cff6bbec2\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2" Jun 10 12:53:38.355087 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:38.355065 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59eb3b55-c728-4a20-bc60-758e5ffc296f-service-ca-bundle\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:38.355353 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:38.355132 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-metrics-certs\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:38.355353 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:38.355178 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 10 12:53:38.355353 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:38.355251 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f57139cf-a599-4dcd-9f59-263cff6bbec2-cluster-monitoring-operator-tls podName:f57139cf-a599-4dcd-9f59-263cff6bbec2 nodeName:}" failed. No retries permitted until 2026-06-10 12:53:46.355230973 +0000 UTC m=+288.095445440 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f57139cf-a599-4dcd-9f59-263cff6bbec2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-j24l2" (UID: "f57139cf-a599-4dcd-9f59-263cff6bbec2") : secret "cluster-monitoring-operator-tls" not found Jun 10 12:53:38.355353 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:38.355262 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 10 12:53:38.355353 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:38.355314 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-metrics-certs podName:59eb3b55-c728-4a20-bc60-758e5ffc296f nodeName:}" failed. No retries permitted until 2026-06-10 12:53:46.355296387 +0000 UTC m=+288.095510861 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-metrics-certs") pod "router-default-6ccc644f47-p5gx4" (UID: "59eb3b55-c728-4a20-bc60-758e5ffc296f") : secret "router-metrics-certs-default" not found Jun 10 12:53:38.355576 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:38.355464 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59eb3b55-c728-4a20-bc60-758e5ffc296f-service-ca-bundle podName:59eb3b55-c728-4a20-bc60-758e5ffc296f nodeName:}" failed. No retries permitted until 2026-06-10 12:53:46.355451335 +0000 UTC m=+288.095665808 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59eb3b55-c728-4a20-bc60-758e5ffc296f-service-ca-bundle") pod "router-default-6ccc644f47-p5gx4" (UID: "59eb3b55-c728-4a20-bc60-758e5ffc296f") : configmap references non-existent config key: service-ca.crt Jun 10 12:53:38.600975 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:38.600950 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jmkgf_de769e15-3af9-4466-8023-7e09fa20df3e/node-ca/0.log" Jun 10 12:53:38.996092 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:38.996067 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bxb7s_c4e62b9a-b59d-4df1-a3fb-8185eb2a5472/serve-healthcheck-canary/0.log" Jun 10 12:53:39.296703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:39.296625 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:39.956583 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:39.956548 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:40.356018 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:40.355940 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:40.609639 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:40.609595 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:40.779413 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:40.779380 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:41.152595 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:41.152565 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:41.217797 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:41.217758 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" podUID="49614bc3-1721-4b77-90b6-8fcfa6db5ae1" Jun 10 12:53:41.217797 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:41.217773 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-b777cbcc5-vb962" podUID="5e7080a1-50fb-441f-989b-c9f2e897f48c" Jun 10 12:53:41.217797 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:41.217786 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-2pwn5" podUID="d0c4887d-c2bc-4f1b-8c0c-29674956c41d" Jun 10 12:53:41.308066 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:41.308029 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:41.518378 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:41.518305 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:53:41.518795 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:41.518305 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2pwn5" Jun 10 12:53:41.518795 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:41.518308 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:53:41.757977 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:41.757944 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:41.860500 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:41.860471 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:42.158223 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:42.158120 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:42.353141 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:42.353109 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:42.558708 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:42.558611 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:42.954648 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:42.954604 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:43.158037 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:43.158003 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:43.756266 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:43.756224 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:44.152498 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:44.152467 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:44.558640 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:44.558519 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:44.669581 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:44.669546 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:44.703832 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:44.703804 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:53:44.703996 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:44.703881 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:53:44.703996 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:44.703926 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-4lhwz\" (UID: \"49614bc3-1721-4b77-90b6-8fcfa6db5ae1\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:53:44.704123 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:44.704067 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 10 12:53:44.704204 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:44.704138 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert podName:49614bc3-1721-4b77-90b6-8fcfa6db5ae1 nodeName:}" failed. No retries permitted until 2026-06-10 12:55:46.704119675 +0000 UTC m=+408.444334135 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-4lhwz" (UID: "49614bc3-1721-4b77-90b6-8fcfa6db5ae1") : secret "networking-console-plugin-cert" not found Jun 10 12:53:44.706221 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:44.706195 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0c4887d-c2bc-4f1b-8c0c-29674956c41d-metrics-tls\") pod \"dns-default-2pwn5\" (UID: \"d0c4887d-c2bc-4f1b-8c0c-29674956c41d\") " pod="openshift-dns/dns-default-2pwn5" Jun 10 12:53:44.706425 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:44.706411 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls\") pod \"image-registry-b777cbcc5-vb962\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:53:44.822889 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:44.822808 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6n4pt\"" Jun 10 12:53:44.822889 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:44.822808 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4f92t\"" Jun 10 12:53:44.829910 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:44.829894 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2pwn5" Jun 10 12:53:44.829990 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:44.829935 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:53:44.955838 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:44.955807 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:44.971739 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:44.971711 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b777cbcc5-vb962"] Jun 10 12:53:44.975508 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:53:44.975473 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e7080a1_50fb_441f_989b_c9f2e897f48c.slice/crio-160a248d8857a89e91bc4d6a3404180c5505076ba5dcf794cf339e0aef3cf54d WatchSource:0}: Error finding container 160a248d8857a89e91bc4d6a3404180c5505076ba5dcf794cf339e0aef3cf54d: Status 404 returned error can't find the container with id 160a248d8857a89e91bc4d6a3404180c5505076ba5dcf794cf339e0aef3cf54d Jun 10 12:53:44.985018 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:44.984997 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2pwn5"] Jun 10 12:53:44.987882 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:53:44.987856 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0c4887d_c2bc_4f1b_8c0c_29674956c41d.slice/crio-b062f8d8e1c655af8e79a6e98a2a289eac2e72a476f2e67995752a838848460c WatchSource:0}: Error finding container b062f8d8e1c655af8e79a6e98a2a289eac2e72a476f2e67995752a838848460c: Status 404 returned error can't find the container with id b062f8d8e1c655af8e79a6e98a2a289eac2e72a476f2e67995752a838848460c Jun 10 12:53:45.157523 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:45.157488 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:45.532148 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:45.532050 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2pwn5" event={"ID":"d0c4887d-c2bc-4f1b-8c0c-29674956c41d","Type":"ContainerStarted","Data":"b062f8d8e1c655af8e79a6e98a2a289eac2e72a476f2e67995752a838848460c"} Jun 10 12:53:45.533719 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:45.533688 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b777cbcc5-vb962" event={"ID":"5e7080a1-50fb-441f-989b-c9f2e897f48c","Type":"ContainerStarted","Data":"1681675b0ad8e9f3da1116a9fe18a001357d171a0d66d97bee1406cac11c6f9a"} Jun 10 12:53:45.533851 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:45.533736 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b777cbcc5-vb962" event={"ID":"5e7080a1-50fb-441f-989b-c9f2e897f48c","Type":"ContainerStarted","Data":"160a248d8857a89e91bc4d6a3404180c5505076ba5dcf794cf339e0aef3cf54d"} Jun 10 12:53:45.533989 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:45.533966 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:53:45.557567 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:45.557501 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-b777cbcc5-vb962" podStartSLOduration=286.557481678 podStartE2EDuration="4m46.557481678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:48:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:53:45.55678743 +0000 UTC m=+287.297001933" watchObservedRunningTime="2026-06-10 12:53:45.557481678 +0000 UTC m=+287.297696162" Jun 10 12:53:45.752761 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:45.752727 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:45.957345 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:45.957298 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:46.158068 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:46.158011 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:46.420515 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:46.420478 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-metrics-certs\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:46.420719 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:46.420553 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f57139cf-a599-4dcd-9f59-263cff6bbec2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-j24l2\" (UID: \"f57139cf-a599-4dcd-9f59-263cff6bbec2\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2" Jun 10 12:53:46.420719 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:46.420609 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59eb3b55-c728-4a20-bc60-758e5ffc296f-service-ca-bundle\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:46.421369 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:46.421341 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59eb3b55-c728-4a20-bc60-758e5ffc296f-service-ca-bundle\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:46.423411 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:46.423392 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59eb3b55-c728-4a20-bc60-758e5ffc296f-metrics-certs\") pod \"router-default-6ccc644f47-p5gx4\" (UID: \"59eb3b55-c728-4a20-bc60-758e5ffc296f\") " pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:46.423508 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:46.423452 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f57139cf-a599-4dcd-9f59-263cff6bbec2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-j24l2\" (UID: \"f57139cf-a599-4dcd-9f59-263cff6bbec2\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2" Jun 10 12:53:46.492293 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:46.492261 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2" Jun 10 12:53:46.498853 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:46.498704 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:46.649294 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:46.649260 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2"] Jun 10 12:53:46.652361 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:53:46.652315 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf57139cf_a599_4dcd_9f59_263cff6bbec2.slice/crio-a6ac0e2ffdd0a3aa85632f59396450f735a705845d99e4f18a1de3d696585dc5 WatchSource:0}: Error finding container a6ac0e2ffdd0a3aa85632f59396450f735a705845d99e4f18a1de3d696585dc5: Status 404 returned error can't find the container with id a6ac0e2ffdd0a3aa85632f59396450f735a705845d99e4f18a1de3d696585dc5 Jun 10 12:53:46.675036 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:46.674974 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6ccc644f47-p5gx4"] Jun 10 12:53:46.677490 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:53:46.677458 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59eb3b55_c728_4a20_bc60_758e5ffc296f.slice/crio-a95686fe3cfbe981d9f99e75fb3d148694c06970b4caf3287c0f4620f288ec76 WatchSource:0}: Error finding container a95686fe3cfbe981d9f99e75fb3d148694c06970b4caf3287c0f4620f288ec76: Status 404 returned error can't find the container with id a95686fe3cfbe981d9f99e75fb3d148694c06970b4caf3287c0f4620f288ec76 Jun 10 12:53:47.357255 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:47.357223 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:47.542872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:47.542832 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2" event={"ID":"f57139cf-a599-4dcd-9f59-263cff6bbec2","Type":"ContainerStarted","Data":"a6ac0e2ffdd0a3aa85632f59396450f735a705845d99e4f18a1de3d696585dc5"} Jun 10 12:53:47.544979 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:47.544838 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2pwn5" event={"ID":"d0c4887d-c2bc-4f1b-8c0c-29674956c41d","Type":"ContainerStarted","Data":"60c6d2c090229acb658d30b29415156293e882f9f8b561a1762d4eb70de4899f"} Jun 10 12:53:47.544979 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:47.544886 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2pwn5" event={"ID":"d0c4887d-c2bc-4f1b-8c0c-29674956c41d","Type":"ContainerStarted","Data":"9b8fb6f627c095a1e32abdd9c34db20633adf09ec92357ae9698cea5e829805c"} Jun 10 12:53:47.544979 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:47.544945 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2pwn5" Jun 10 12:53:47.546826 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:47.546761 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6ccc644f47-p5gx4" event={"ID":"59eb3b55-c728-4a20-bc60-758e5ffc296f","Type":"ContainerStarted","Data":"5c22a34f8b3f20bebdc11fde889d0a68da6125f570d9e53d968fb4ce2927a675"} Jun 10 12:53:47.546826 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:47.546804 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6ccc644f47-p5gx4" event={"ID":"59eb3b55-c728-4a20-bc60-758e5ffc296f","Type":"ContainerStarted","Data":"a95686fe3cfbe981d9f99e75fb3d148694c06970b4caf3287c0f4620f288ec76"} Jun 10 12:53:47.559519 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:47.559444 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:47.568726 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:47.568688 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2pwn5" podStartSLOduration=252.058859066 podStartE2EDuration="4m13.568676341s" totalImagesPullingTime="1.509817275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:49:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:53:47.568511814 +0000 UTC m=+289.308726299" watchObservedRunningTime="2026-06-10 12:53:47.568676341 +0000 UTC m=+289.308890822" Jun 10 12:53:47.594517 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:47.594463 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6ccc644f47-p5gx4" podStartSLOduration=17.594445776 podStartE2EDuration="17.594445776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:53:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:53:47.594037962 +0000 UTC m=+289.334252444" watchObservedRunningTime="2026-06-10 12:53:47.594445776 +0000 UTC m=+289.334660237" Jun 10 12:53:47.758571 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:47.758481 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:47.895690 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:47.895655 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:47.997004 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:47.996973 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:48.150539 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:48.150510 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:48.500270 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:48.500237 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:48.502264 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:48.502031 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:48.551691 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:48.551660 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2" event={"ID":"f57139cf-a599-4dcd-9f59-263cff6bbec2","Type":"ContainerStarted","Data":"075c3ce95b799da91605938d157d91cd48c90f5d2a596cc4dfed3b30758c5fa1"} Jun 10 12:53:48.551874 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:48.551844 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:48.553105 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:48.553084 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6ccc644f47-p5gx4" Jun 10 12:53:48.570277 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:48.570240 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-j24l2" podStartSLOduration=16.860386156 podStartE2EDuration="18.570226342s" totalImagesPullingTime="1.709840186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:53:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:53:48.569949047 +0000 UTC m=+290.310163527" watchObservedRunningTime="2026-06-10 12:53:48.570226342 +0000 UTC m=+290.310440818" Jun 10 12:53:48.695153 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:48.695069 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:48.796543 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:48.796488 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:48.898123 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:48.898086 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:48.998034 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:48.997941 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:49.061536 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:49.061499 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:49.355967 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:49.355872 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:52.713357 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:53:52.713320 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-2xw2r" podUID="080cea4a-0afb-41b3-a04c-5763fbb54eac" Jun 10 12:53:55.269317 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:55.269258 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:55.333665 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:55.333632 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:56.734948 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:56.734908 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:56.836676 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:56.836647 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:56.896064 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:56.896026 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:57.033494 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:57.033393 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:57.228395 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:57.228350 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:57.292576 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:57.292482 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:57.554009 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:57.553923 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2pwn5" Jun 10 12:53:57.615393 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:57.615341 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:57.679266 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:57.679232 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:58.038232 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.038188 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:58.119646 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.119616 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:58.184944 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.184916 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pbn64"] Jun 10 12:53:58.189192 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.189158 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pbn64" Jun 10 12:53:58.192002 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.191974 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 10 12:53:58.192002 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.191998 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 10 12:53:58.192186 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.192005 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 10 12:53:58.193282 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.193265 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7zsvb\"" Jun 10 12:53:58.193409 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.193388 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 10 12:53:58.202659 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.202639 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pbn64"] Jun 10 12:53:58.318123 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.318043 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c50085d0-35f4-40c0-8048-69b018c82b63-data-volume\") pod \"insights-runtime-extractor-pbn64\" (UID: \"c50085d0-35f4-40c0-8048-69b018c82b63\") " pod="openshift-insights/insights-runtime-extractor-pbn64" Jun 10 12:53:58.318123 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.318079 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk6rm\" (UniqueName: \"kubernetes.io/projected/c50085d0-35f4-40c0-8048-69b018c82b63-kube-api-access-kk6rm\") pod \"insights-runtime-extractor-pbn64\" (UID: \"c50085d0-35f4-40c0-8048-69b018c82b63\") " pod="openshift-insights/insights-runtime-extractor-pbn64" Jun 10 12:53:58.318362 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.318209 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c50085d0-35f4-40c0-8048-69b018c82b63-crio-socket\") pod \"insights-runtime-extractor-pbn64\" (UID: \"c50085d0-35f4-40c0-8048-69b018c82b63\") " pod="openshift-insights/insights-runtime-extractor-pbn64" Jun 10 12:53:58.318362 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.318240 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c50085d0-35f4-40c0-8048-69b018c82b63-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pbn64\" (UID: \"c50085d0-35f4-40c0-8048-69b018c82b63\") " pod="openshift-insights/insights-runtime-extractor-pbn64" Jun 10 12:53:58.318362 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.318266 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c50085d0-35f4-40c0-8048-69b018c82b63-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pbn64\" (UID: \"c50085d0-35f4-40c0-8048-69b018c82b63\") " pod="openshift-insights/insights-runtime-extractor-pbn64" Jun 10 12:53:58.393895 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.393856 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:58.418826 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.418795 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c50085d0-35f4-40c0-8048-69b018c82b63-crio-socket\") pod \"insights-runtime-extractor-pbn64\" (UID: \"c50085d0-35f4-40c0-8048-69b018c82b63\") " pod="openshift-insights/insights-runtime-extractor-pbn64" Jun 10 12:53:58.418959 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.418829 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c50085d0-35f4-40c0-8048-69b018c82b63-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pbn64\" (UID: \"c50085d0-35f4-40c0-8048-69b018c82b63\") " pod="openshift-insights/insights-runtime-extractor-pbn64" Jun 10 12:53:58.418959 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.418882 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c50085d0-35f4-40c0-8048-69b018c82b63-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pbn64\" (UID: \"c50085d0-35f4-40c0-8048-69b018c82b63\") " pod="openshift-insights/insights-runtime-extractor-pbn64" Jun 10 12:53:58.418959 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.418910 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c50085d0-35f4-40c0-8048-69b018c82b63-crio-socket\") pod \"insights-runtime-extractor-pbn64\" (UID: \"c50085d0-35f4-40c0-8048-69b018c82b63\") " pod="openshift-insights/insights-runtime-extractor-pbn64" Jun 10 12:53:58.418959 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.418923 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c50085d0-35f4-40c0-8048-69b018c82b63-data-volume\") pod \"insights-runtime-extractor-pbn64\" (UID: \"c50085d0-35f4-40c0-8048-69b018c82b63\") " pod="openshift-insights/insights-runtime-extractor-pbn64" Jun 10 12:53:58.419099 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.419085 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk6rm\" (UniqueName: \"kubernetes.io/projected/c50085d0-35f4-40c0-8048-69b018c82b63-kube-api-access-kk6rm\") pod \"insights-runtime-extractor-pbn64\" (UID: \"c50085d0-35f4-40c0-8048-69b018c82b63\") " pod="openshift-insights/insights-runtime-extractor-pbn64" Jun 10 12:53:58.419311 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.419290 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c50085d0-35f4-40c0-8048-69b018c82b63-data-volume\") pod \"insights-runtime-extractor-pbn64\" (UID: \"c50085d0-35f4-40c0-8048-69b018c82b63\") " pod="openshift-insights/insights-runtime-extractor-pbn64" Jun 10 12:53:58.419476 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.419461 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c50085d0-35f4-40c0-8048-69b018c82b63-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pbn64\" (UID: \"c50085d0-35f4-40c0-8048-69b018c82b63\") " pod="openshift-insights/insights-runtime-extractor-pbn64" Jun 10 12:53:58.421357 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.421337 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c50085d0-35f4-40c0-8048-69b018c82b63-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pbn64\" (UID: \"c50085d0-35f4-40c0-8048-69b018c82b63\") " pod="openshift-insights/insights-runtime-extractor-pbn64" Jun 10 12:53:58.432584 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.432565 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk6rm\" (UniqueName: \"kubernetes.io/projected/c50085d0-35f4-40c0-8048-69b018c82b63-kube-api-access-kk6rm\") pod \"insights-runtime-extractor-pbn64\" (UID: \"c50085d0-35f4-40c0-8048-69b018c82b63\") " pod="openshift-insights/insights-runtime-extractor-pbn64" Jun 10 12:53:58.487064 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.487025 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:58.498760 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.498736 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pbn64" Jun 10 12:53:58.636487 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.636408 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pbn64"] Jun 10 12:53:58.637579 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.637552 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:58.639228 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:53:58.639207 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc50085d0_35f4_40c0_8048_69b018c82b63.slice/crio-733de2d0eda4f84914b1ee75dfa46d20b90a51cb18d7ec1edbda16b849dd5992 WatchSource:0}: Error finding container 733de2d0eda4f84914b1ee75dfa46d20b90a51cb18d7ec1edbda16b849dd5992: Status 404 returned error can't find the container with id 733de2d0eda4f84914b1ee75dfa46d20b90a51cb18d7ec1edbda16b849dd5992 Jun 10 12:53:58.670326 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.670305 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgg5q_8a91de05-d0b9-4192-9583-8de8cf91c175/ovn-acl-logging/0.log" Jun 10 12:53:58.671558 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.671530 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgg5q_8a91de05-d0b9-4192-9583-8de8cf91c175/ovn-acl-logging/0.log" Jun 10 12:53:58.674839 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.674822 2581 kubelet.go:1648] "Image garbage collection succeeded" Jun 10 12:53:58.844553 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.844365 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:58.998477 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:58.998434 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:59.447129 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:59.447095 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:59.548129 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:59.548040 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:53:59.587112 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:59.587083 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pbn64" event={"ID":"c50085d0-35f4-40c0-8048-69b018c82b63","Type":"ContainerStarted","Data":"d455a0103ca3cad4b0f3832b9e8172040845ca97dd4aab983d1a61d0fd5d82d7"} Jun 10 12:53:59.587244 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:59.587162 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pbn64" event={"ID":"c50085d0-35f4-40c0-8048-69b018c82b63","Type":"ContainerStarted","Data":"064007edda1768a76591e6536980faa2657333083e21cdc7f1e904d9975542c0"} Jun 10 12:53:59.587244 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:53:59.587205 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pbn64" event={"ID":"c50085d0-35f4-40c0-8048-69b018c82b63","Type":"ContainerStarted","Data":"733de2d0eda4f84914b1ee75dfa46d20b90a51cb18d7ec1edbda16b849dd5992"} Jun 10 12:54:00.917069 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:00.917037 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:00.953739 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:00.953709 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj"] Jun 10 12:54:00.957034 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:00.957018 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj" Jun 10 12:54:00.959656 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:00.959639 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 10 12:54:00.959743 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:00.959639 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 10 12:54:00.960762 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:00.960742 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-tfnhp\"" Jun 10 12:54:00.960881 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:00.960820 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 10 12:54:00.966146 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:00.966125 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj"] Jun 10 12:54:00.984049 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:00.984025 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:01.040092 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:01.040006 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d1cad173-0816-45cf-a41f-91efbe0fc87f-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-s7lkj\" (UID: \"d1cad173-0816-45cf-a41f-91efbe0fc87f\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj" Jun 10 12:54:01.040092 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:01.040048 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1cad173-0816-45cf-a41f-91efbe0fc87f-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-s7lkj\" (UID: \"d1cad173-0816-45cf-a41f-91efbe0fc87f\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj" Jun 10 12:54:01.040291 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:01.040100 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nbq5\" (UniqueName: \"kubernetes.io/projected/d1cad173-0816-45cf-a41f-91efbe0fc87f-kube-api-access-9nbq5\") pod \"prometheus-operator-7f7d445d84-s7lkj\" (UID: \"d1cad173-0816-45cf-a41f-91efbe0fc87f\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj" Jun 10 12:54:01.040291 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:01.040136 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d1cad173-0816-45cf-a41f-91efbe0fc87f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-s7lkj\" (UID: \"d1cad173-0816-45cf-a41f-91efbe0fc87f\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj" Jun 10 12:54:01.140862 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:01.140824 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d1cad173-0816-45cf-a41f-91efbe0fc87f-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-s7lkj\" (UID: \"d1cad173-0816-45cf-a41f-91efbe0fc87f\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj" Jun 10 12:54:01.140992 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:01.140873 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1cad173-0816-45cf-a41f-91efbe0fc87f-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-s7lkj\" (UID: \"d1cad173-0816-45cf-a41f-91efbe0fc87f\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj" Jun 10 12:54:01.140992 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:01.140904 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nbq5\" (UniqueName: \"kubernetes.io/projected/d1cad173-0816-45cf-a41f-91efbe0fc87f-kube-api-access-9nbq5\") pod \"prometheus-operator-7f7d445d84-s7lkj\" (UID: \"d1cad173-0816-45cf-a41f-91efbe0fc87f\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj" Jun 10 12:54:01.140992 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:01.140935 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d1cad173-0816-45cf-a41f-91efbe0fc87f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-s7lkj\" (UID: \"d1cad173-0816-45cf-a41f-91efbe0fc87f\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj" Jun 10 12:54:01.140992 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:54:01.140968 2581 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jun 10 12:54:01.141143 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:54:01.141036 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1cad173-0816-45cf-a41f-91efbe0fc87f-prometheus-operator-tls podName:d1cad173-0816-45cf-a41f-91efbe0fc87f nodeName:}" failed. No retries permitted until 2026-06-10 12:54:01.641015247 +0000 UTC m=+303.381229706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/d1cad173-0816-45cf-a41f-91efbe0fc87f-prometheus-operator-tls") pod "prometheus-operator-7f7d445d84-s7lkj" (UID: "d1cad173-0816-45cf-a41f-91efbe0fc87f") : secret "prometheus-operator-tls" not found Jun 10 12:54:01.141627 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:01.141601 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1cad173-0816-45cf-a41f-91efbe0fc87f-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-s7lkj\" (UID: \"d1cad173-0816-45cf-a41f-91efbe0fc87f\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj" Jun 10 12:54:01.143524 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:01.143503 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d1cad173-0816-45cf-a41f-91efbe0fc87f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-s7lkj\" (UID: \"d1cad173-0816-45cf-a41f-91efbe0fc87f\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj" Jun 10 12:54:01.150794 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:01.150772 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nbq5\" (UniqueName: \"kubernetes.io/projected/d1cad173-0816-45cf-a41f-91efbe0fc87f-kube-api-access-9nbq5\") pod \"prometheus-operator-7f7d445d84-s7lkj\" (UID: \"d1cad173-0816-45cf-a41f-91efbe0fc87f\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj" Jun 10 12:54:01.327136 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:01.327060 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:01.596038 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:01.595957 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pbn64" event={"ID":"c50085d0-35f4-40c0-8048-69b018c82b63","Type":"ContainerStarted","Data":"9765ecd4be23df89ad75acb3cd1e7df3252ee917ab43e5e28e82b594568fe7ba"} Jun 10 12:54:01.618525 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:01.618463 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pbn64" podStartSLOduration=1.623575233 podStartE2EDuration="3.618447825s" totalImagesPullingTime="1.994872592s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 12:53:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:54:01.617679437 +0000 UTC m=+303.357893918" watchObservedRunningTime="2026-06-10 12:54:01.618447825 +0000 UTC m=+303.358662335" Jun 10 12:54:01.645254 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:01.645228 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d1cad173-0816-45cf-a41f-91efbe0fc87f-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-s7lkj\" (UID: \"d1cad173-0816-45cf-a41f-91efbe0fc87f\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj" Jun 10 12:54:01.647753 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:01.647729 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d1cad173-0816-45cf-a41f-91efbe0fc87f-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-s7lkj\" (UID: \"d1cad173-0816-45cf-a41f-91efbe0fc87f\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj" Jun 10 12:54:01.866680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:01.866658 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj" Jun 10 12:54:01.997502 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:01.997471 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj"] Jun 10 12:54:02.000466 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:54:02.000438 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1cad173_0816_45cf_a41f_91efbe0fc87f.slice/crio-b227bc3eac8703a67f2679fb0e546883a4c693378412a5e55ff77e894d04732b WatchSource:0}: Error finding container b227bc3eac8703a67f2679fb0e546883a4c693378412a5e55ff77e894d04732b: Status 404 returned error can't find the container with id b227bc3eac8703a67f2679fb0e546883a4c693378412a5e55ff77e894d04732b Jun 10 12:54:02.002453 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:02.002437 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 12:54:02.600801 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:02.600762 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj" event={"ID":"d1cad173-0816-45cf-a41f-91efbe0fc87f","Type":"ContainerStarted","Data":"b227bc3eac8703a67f2679fb0e546883a4c693378412a5e55ff77e894d04732b"} Jun 10 12:54:03.489999 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:03.489905 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:03.609455 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:03.609420 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj" event={"ID":"d1cad173-0816-45cf-a41f-91efbe0fc87f","Type":"ContainerStarted","Data":"228655f194909ee169d21de26f7484b37ddb8fc55402be3196a0f7d5b9474234"} Jun 10 12:54:03.609623 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:03.609492 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj" event={"ID":"d1cad173-0816-45cf-a41f-91efbe0fc87f","Type":"ContainerStarted","Data":"d617897f72cc13342f747aa3002c874fec065cf3b47bffa2c156e9ffb11c464d"} Jun 10 12:54:03.628818 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:03.628772 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7f7d445d84-s7lkj" podStartSLOduration=2.417652945 podStartE2EDuration="3.628758511s" totalImagesPullingTime="1.211105566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:54:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:54:03.626896613 +0000 UTC m=+305.367111093" watchObservedRunningTime="2026-06-10 12:54:03.628758511 +0000 UTC m=+305.368972993" Jun 10 12:54:04.665679 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:04.665643 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:04.724072 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:04.724037 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:04.835354 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:04.835306 2581 patch_prober.go:28] interesting pod/image-registry-b777cbcc5-vb962 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 10 12:54:04.835503 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:04.835364 2581 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-b777cbcc5-vb962" podUID="5e7080a1-50fb-441f-989b-c9f2e897f48c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 10 12:54:05.071467 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.071384 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:05.303887 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.303851 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:05.333857 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.333792 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75"] Jun 10 12:54:05.337627 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.337598 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" Jun 10 12:54:05.340406 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.340380 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-rwv99\"" Jun 10 12:54:05.340764 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.340428 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 10 12:54:05.340847 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.340438 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 10 12:54:05.349614 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.349591 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nqh9k"] Jun 10 12:54:05.352924 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.352905 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75"] Jun 10 12:54:05.353064 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.353038 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.359677 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.359659 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 10 12:54:05.360509 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.359841 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 10 12:54:05.360760 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.359901 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 10 12:54:05.360983 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.359954 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-675mt\"" Jun 10 12:54:05.411492 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.411454 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:05.476239 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.476205 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bdc3c6b-9206-495d-a0a8-142384bfa823-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.476401 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.476286 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93cd4396-77ee-40b7-be1f-79e58bb2fae5-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-krz75\" (UID: \"93cd4396-77ee-40b7-be1f-79e58bb2fae5\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" Jun 10 12:54:05.476401 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.476315 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2bdc3c6b-9206-495d-a0a8-142384bfa823-sys\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.476401 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.476361 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93cd4396-77ee-40b7-be1f-79e58bb2fae5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-krz75\" (UID: \"93cd4396-77ee-40b7-be1f-79e58bb2fae5\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" Jun 10 12:54:05.476401 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.476382 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2bdc3c6b-9206-495d-a0a8-142384bfa823-node-exporter-wtmp\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.476604 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.476412 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2bdc3c6b-9206-495d-a0a8-142384bfa823-node-exporter-textfile\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.476604 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.476434 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2bdc3c6b-9206-495d-a0a8-142384bfa823-node-exporter-tls\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.476604 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.476471 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2bdc3c6b-9206-495d-a0a8-142384bfa823-node-exporter-accelerators-collector-config\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.476604 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.476530 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t7nn\" (UniqueName: \"kubernetes.io/projected/2bdc3c6b-9206-495d-a0a8-142384bfa823-kube-api-access-4t7nn\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.476604 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.476584 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/93cd4396-77ee-40b7-be1f-79e58bb2fae5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-krz75\" (UID: \"93cd4396-77ee-40b7-be1f-79e58bb2fae5\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" Jun 10 12:54:05.476835 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.476606 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:05.476835 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.476639 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2bdc3c6b-9206-495d-a0a8-142384bfa823-root\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.476835 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.476694 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bdc3c6b-9206-495d-a0a8-142384bfa823-metrics-client-ca\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.476835 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.476744 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl8w7\" (UniqueName: \"kubernetes.io/projected/93cd4396-77ee-40b7-be1f-79e58bb2fae5-kube-api-access-kl8w7\") pod \"openshift-state-metrics-65f78d5c66-krz75\" (UID: \"93cd4396-77ee-40b7-be1f-79e58bb2fae5\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" Jun 10 12:54:05.577460 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.577435 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/93cd4396-77ee-40b7-be1f-79e58bb2fae5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-krz75\" (UID: \"93cd4396-77ee-40b7-be1f-79e58bb2fae5\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" Jun 10 12:54:05.577634 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.577482 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2bdc3c6b-9206-495d-a0a8-142384bfa823-root\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.577634 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.577499 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bdc3c6b-9206-495d-a0a8-142384bfa823-metrics-client-ca\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.577634 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.577543 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2bdc3c6b-9206-495d-a0a8-142384bfa823-root\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.577634 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:54:05.577593 2581 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jun 10 12:54:05.577634 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.577623 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl8w7\" (UniqueName: \"kubernetes.io/projected/93cd4396-77ee-40b7-be1f-79e58bb2fae5-kube-api-access-kl8w7\") pod \"openshift-state-metrics-65f78d5c66-krz75\" (UID: \"93cd4396-77ee-40b7-be1f-79e58bb2fae5\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" Jun 10 12:54:05.577877 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:54:05.577652 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93cd4396-77ee-40b7-be1f-79e58bb2fae5-openshift-state-metrics-tls podName:93cd4396-77ee-40b7-be1f-79e58bb2fae5 nodeName:}" failed. No retries permitted until 2026-06-10 12:54:06.077632027 +0000 UTC m=+307.817846493 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/93cd4396-77ee-40b7-be1f-79e58bb2fae5-openshift-state-metrics-tls") pod "openshift-state-metrics-65f78d5c66-krz75" (UID: "93cd4396-77ee-40b7-be1f-79e58bb2fae5") : secret "openshift-state-metrics-tls" not found Jun 10 12:54:05.577877 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.577686 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bdc3c6b-9206-495d-a0a8-142384bfa823-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.577877 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.577718 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93cd4396-77ee-40b7-be1f-79e58bb2fae5-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-krz75\" (UID: \"93cd4396-77ee-40b7-be1f-79e58bb2fae5\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" Jun 10 12:54:05.577877 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.577744 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2bdc3c6b-9206-495d-a0a8-142384bfa823-sys\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.577877 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.577797 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93cd4396-77ee-40b7-be1f-79e58bb2fae5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-krz75\" (UID: \"93cd4396-77ee-40b7-be1f-79e58bb2fae5\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" Jun 10 12:54:05.577877 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.577833 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2bdc3c6b-9206-495d-a0a8-142384bfa823-node-exporter-wtmp\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.577877 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.577874 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2bdc3c6b-9206-495d-a0a8-142384bfa823-node-exporter-textfile\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.578257 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.577878 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2bdc3c6b-9206-495d-a0a8-142384bfa823-sys\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.578257 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.577897 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2bdc3c6b-9206-495d-a0a8-142384bfa823-node-exporter-tls\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.578257 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.577931 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2bdc3c6b-9206-495d-a0a8-142384bfa823-node-exporter-accelerators-collector-config\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.578257 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.577960 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t7nn\" (UniqueName: \"kubernetes.io/projected/2bdc3c6b-9206-495d-a0a8-142384bfa823-kube-api-access-4t7nn\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.578257 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.578012 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2bdc3c6b-9206-495d-a0a8-142384bfa823-node-exporter-wtmp\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.578257 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.578223 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2bdc3c6b-9206-495d-a0a8-142384bfa823-node-exporter-textfile\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.578686 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.578628 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93cd4396-77ee-40b7-be1f-79e58bb2fae5-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-krz75\" (UID: \"93cd4396-77ee-40b7-be1f-79e58bb2fae5\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" Jun 10 12:54:05.579261 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.578914 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2bdc3c6b-9206-495d-a0a8-142384bfa823-node-exporter-accelerators-collector-config\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.579261 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.578938 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bdc3c6b-9206-495d-a0a8-142384bfa823-metrics-client-ca\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.580663 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.580642 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2bdc3c6b-9206-495d-a0a8-142384bfa823-node-exporter-tls\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.580773 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.580751 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93cd4396-77ee-40b7-be1f-79e58bb2fae5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-krz75\" (UID: \"93cd4396-77ee-40b7-be1f-79e58bb2fae5\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" Jun 10 12:54:05.580820 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.580804 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bdc3c6b-9206-495d-a0a8-142384bfa823-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.587363 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.587254 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:05.592896 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.592878 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl8w7\" (UniqueName: \"kubernetes.io/projected/93cd4396-77ee-40b7-be1f-79e58bb2fae5-kube-api-access-kl8w7\") pod \"openshift-state-metrics-65f78d5c66-krz75\" (UID: \"93cd4396-77ee-40b7-be1f-79e58bb2fae5\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" Jun 10 12:54:05.595226 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.595208 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t7nn\" (UniqueName: \"kubernetes.io/projected/2bdc3c6b-9206-495d-a0a8-142384bfa823-kube-api-access-4t7nn\") pod \"node-exporter-nqh9k\" (UID: \"2bdc3c6b-9206-495d-a0a8-142384bfa823\") " pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.667585 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.667553 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nqh9k" Jun 10 12:54:05.680675 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:54:05.680628 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bdc3c6b_9206_495d_a0a8_142384bfa823.slice/crio-b6a7b6ef009090af1416faf7255db57e3b1bad7c9bbad55b9bd1ebd9a3f8080a WatchSource:0}: Error finding container b6a7b6ef009090af1416faf7255db57e3b1bad7c9bbad55b9bd1ebd9a3f8080a: Status 404 returned error can't find the container with id b6a7b6ef009090af1416faf7255db57e3b1bad7c9bbad55b9bd1ebd9a3f8080a Jun 10 12:54:05.705540 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.705506 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:05.872608 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.872578 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d896d57f5-t9t95"] Jun 10 12:54:05.876625 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.876602 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:05.879652 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.879623 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 10 12:54:05.880219 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.879990 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 10 12:54:05.880219 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.880001 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 10 12:54:05.880219 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.880011 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 10 12:54:05.880219 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.880052 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 10 12:54:05.880219 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.880005 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 10 12:54:05.880219 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.880054 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 10 12:54:05.880219 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.880214 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-j2fn2\"" Jun 10 12:54:05.889230 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.889211 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d896d57f5-t9t95"] Jun 10 12:54:05.982941 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.982905 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f8d53f03-d7d2-439d-97e4-d19aae54d880-console-oauth-config\") pod \"console-6d896d57f5-t9t95\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:05.983122 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.982954 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f8d53f03-d7d2-439d-97e4-d19aae54d880-service-ca\") pod \"console-6d896d57f5-t9t95\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:05.983122 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.982995 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phhjd\" (UniqueName: \"kubernetes.io/projected/f8d53f03-d7d2-439d-97e4-d19aae54d880-kube-api-access-phhjd\") pod \"console-6d896d57f5-t9t95\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:05.983122 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.983044 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f8d53f03-d7d2-439d-97e4-d19aae54d880-console-config\") pod \"console-6d896d57f5-t9t95\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:05.983315 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.983141 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f8d53f03-d7d2-439d-97e4-d19aae54d880-console-serving-cert\") pod \"console-6d896d57f5-t9t95\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:05.983315 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:05.983217 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f8d53f03-d7d2-439d-97e4-d19aae54d880-oauth-serving-cert\") pod \"console-6d896d57f5-t9t95\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:06.082413 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.082374 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:06.084306 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.084281 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f8d53f03-d7d2-439d-97e4-d19aae54d880-console-oauth-config\") pod \"console-6d896d57f5-t9t95\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:06.084437 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.084323 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f8d53f03-d7d2-439d-97e4-d19aae54d880-service-ca\") pod \"console-6d896d57f5-t9t95\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:06.084437 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.084362 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phhjd\" (UniqueName: \"kubernetes.io/projected/f8d53f03-d7d2-439d-97e4-d19aae54d880-kube-api-access-phhjd\") pod \"console-6d896d57f5-t9t95\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:06.084437 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.084415 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f8d53f03-d7d2-439d-97e4-d19aae54d880-console-config\") pod \"console-6d896d57f5-t9t95\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:06.084594 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.084455 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/93cd4396-77ee-40b7-be1f-79e58bb2fae5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-krz75\" (UID: \"93cd4396-77ee-40b7-be1f-79e58bb2fae5\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" Jun 10 12:54:06.084594 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.084481 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f8d53f03-d7d2-439d-97e4-d19aae54d880-console-serving-cert\") pod \"console-6d896d57f5-t9t95\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:06.084594 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.084527 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f8d53f03-d7d2-439d-97e4-d19aae54d880-oauth-serving-cert\") pod \"console-6d896d57f5-t9t95\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:06.084787 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:54:06.084763 2581 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jun 10 12:54:06.084848 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:54:06.084833 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93cd4396-77ee-40b7-be1f-79e58bb2fae5-openshift-state-metrics-tls podName:93cd4396-77ee-40b7-be1f-79e58bb2fae5 nodeName:}" failed. No retries permitted until 2026-06-10 12:54:07.084814029 +0000 UTC m=+308.825028505 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/93cd4396-77ee-40b7-be1f-79e58bb2fae5-openshift-state-metrics-tls") pod "openshift-state-metrics-65f78d5c66-krz75" (UID: "93cd4396-77ee-40b7-be1f-79e58bb2fae5") : secret "openshift-state-metrics-tls" not found Jun 10 12:54:06.085288 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.085088 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f8d53f03-d7d2-439d-97e4-d19aae54d880-service-ca\") pod \"console-6d896d57f5-t9t95\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:06.085288 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.085221 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f8d53f03-d7d2-439d-97e4-d19aae54d880-oauth-serving-cert\") pod \"console-6d896d57f5-t9t95\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:06.085461 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.085403 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f8d53f03-d7d2-439d-97e4-d19aae54d880-console-config\") pod \"console-6d896d57f5-t9t95\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:06.087244 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.087222 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f8d53f03-d7d2-439d-97e4-d19aae54d880-console-oauth-config\") pod \"console-6d896d57f5-t9t95\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:06.087389 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.087371 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f8d53f03-d7d2-439d-97e4-d19aae54d880-console-serving-cert\") pod \"console-6d896d57f5-t9t95\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:06.093708 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.093688 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phhjd\" (UniqueName: \"kubernetes.io/projected/f8d53f03-d7d2-439d-97e4-d19aae54d880-kube-api-access-phhjd\") pod \"console-6d896d57f5-t9t95\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:06.181285 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.181197 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:06.187152 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.187115 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:06.330276 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.330241 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:06.348415 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.348383 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d896d57f5-t9t95"] Jun 10 12:54:06.354073 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:54:06.354029 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8d53f03_d7d2_439d_97e4_d19aae54d880.slice/crio-94b1aa8fa5af02941dbea0760f7395fbad46ccfda508c53e54e899c30b003fb0 WatchSource:0}: Error finding container 94b1aa8fa5af02941dbea0760f7395fbad46ccfda508c53e54e899c30b003fb0: Status 404 returned error can't find the container with id 94b1aa8fa5af02941dbea0760f7395fbad46ccfda508c53e54e899c30b003fb0 Jun 10 12:54:06.433579 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.433498 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:06.443073 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.443048 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 10 12:54:06.448611 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.448588 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.451706 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.451687 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 10 12:54:06.452034 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.451912 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 10 12:54:06.452300 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.452277 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 10 12:54:06.452454 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.452430 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 10 12:54:06.452656 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.452632 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 10 12:54:06.452735 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.452688 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 10 12:54:06.452823 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.452639 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 10 12:54:06.452946 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.452928 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qfwxh\"" Jun 10 12:54:06.453197 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.453147 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 10 12:54:06.456355 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.455876 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 10 12:54:06.468249 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.468224 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 10 12:54:06.542594 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.542542 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:54:06.588679 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.588644 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.588863 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.588699 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-config-volume\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.588863 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.588730 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/45e8ee35-83b9-416a-98a9-45ddfb54437e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.588863 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.588764 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.588863 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.588803 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45e8ee35-83b9-416a-98a9-45ddfb54437e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.588863 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.588849 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/45e8ee35-83b9-416a-98a9-45ddfb54437e-config-out\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.589113 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.588872 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/45e8ee35-83b9-416a-98a9-45ddfb54437e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.589113 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.588901 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-web-config\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.589113 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.588995 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.589113 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.589022 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.589113 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.589046 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45e8ee35-83b9-416a-98a9-45ddfb54437e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.589113 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.589073 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.589463 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.589116 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rhjs\" (UniqueName: \"kubernetes.io/projected/45e8ee35-83b9-416a-98a9-45ddfb54437e-kube-api-access-8rhjs\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.621555 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.621488 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d896d57f5-t9t95" event={"ID":"f8d53f03-d7d2-439d-97e4-d19aae54d880","Type":"ContainerStarted","Data":"94b1aa8fa5af02941dbea0760f7395fbad46ccfda508c53e54e899c30b003fb0"} Jun 10 12:54:06.623071 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.623043 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nqh9k" event={"ID":"2bdc3c6b-9206-495d-a0a8-142384bfa823","Type":"ContainerStarted","Data":"b6a7b6ef009090af1416faf7255db57e3b1bad7c9bbad55b9bd1ebd9a3f8080a"} Jun 10 12:54:06.692928 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.690428 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.692928 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.690472 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.692928 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.690496 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45e8ee35-83b9-416a-98a9-45ddfb54437e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.692928 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.690521 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.692928 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.690557 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rhjs\" (UniqueName: \"kubernetes.io/projected/45e8ee35-83b9-416a-98a9-45ddfb54437e-kube-api-access-8rhjs\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.692928 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.690590 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.692928 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.690630 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-config-volume\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.692928 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.690653 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/45e8ee35-83b9-416a-98a9-45ddfb54437e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.692928 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.690686 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.692928 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.690725 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45e8ee35-83b9-416a-98a9-45ddfb54437e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.692928 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.690780 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/45e8ee35-83b9-416a-98a9-45ddfb54437e-config-out\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.692928 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.690800 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/45e8ee35-83b9-416a-98a9-45ddfb54437e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.692928 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.690826 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-web-config\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.693945 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.692941 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/45e8ee35-83b9-416a-98a9-45ddfb54437e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.694389 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.694332 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45e8ee35-83b9-416a-98a9-45ddfb54437e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.698299 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.695612 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-config-volume\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.698299 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.697057 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45e8ee35-83b9-416a-98a9-45ddfb54437e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.700156 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.698915 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/45e8ee35-83b9-416a-98a9-45ddfb54437e-config-out\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.700156 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.699354 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.700156 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.699419 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/45e8ee35-83b9-416a-98a9-45ddfb54437e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.700156 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.699629 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.700430 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.700200 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.700673 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.700619 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-web-config\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.700673 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.700645 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.700950 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.700929 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.706755 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.706736 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rhjs\" (UniqueName: \"kubernetes.io/projected/45e8ee35-83b9-416a-98a9-45ddfb54437e-kube-api-access-8rhjs\") pod \"alertmanager-main-0\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.709233 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.709155 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:06.769832 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.769809 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:54:06.910606 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.910578 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:06.924085 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.924049 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 10 12:54:06.928200 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:54:06.928160 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45e8ee35_83b9_416a_98a9_45ddfb54437e.slice/crio-ca20e362a6f2690c1286d0c90f189cd480a9eed70739716efd82e3e39f26c9b8 WatchSource:0}: Error finding container ca20e362a6f2690c1286d0c90f189cd480a9eed70739716efd82e3e39f26c9b8: Status 404 returned error can't find the container with id ca20e362a6f2690c1286d0c90f189cd480a9eed70739716efd82e3e39f26c9b8 Jun 10 12:54:06.971531 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:06.971454 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:07.094605 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:07.094564 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/93cd4396-77ee-40b7-be1f-79e58bb2fae5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-krz75\" (UID: \"93cd4396-77ee-40b7-be1f-79e58bb2fae5\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" Jun 10 12:54:07.097494 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:07.097469 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/93cd4396-77ee-40b7-be1f-79e58bb2fae5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-krz75\" (UID: \"93cd4396-77ee-40b7-be1f-79e58bb2fae5\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" Jun 10 12:54:07.151518 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:07.151490 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" Jun 10 12:54:07.308208 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:07.308095 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75"] Jun 10 12:54:07.311101 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:54:07.311070 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93cd4396_77ee_40b7_be1f_79e58bb2fae5.slice/crio-0f6024a3df4ab7694ec98554e9674b6b915b18e7c153bd085a6e5892a2de77bf WatchSource:0}: Error finding container 0f6024a3df4ab7694ec98554e9674b6b915b18e7c153bd085a6e5892a2de77bf: Status 404 returned error can't find the container with id 0f6024a3df4ab7694ec98554e9674b6b915b18e7c153bd085a6e5892a2de77bf Jun 10 12:54:07.317594 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:07.317570 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:07.384411 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:07.384385 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:07.630056 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:07.630015 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" event={"ID":"93cd4396-77ee-40b7-be1f-79e58bb2fae5","Type":"ContainerStarted","Data":"2b9e4f6f7be29a52b995a2473c6f7551828dfea7d7058ad9afff7fbb14902b88"} Jun 10 12:54:07.630252 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:07.630095 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" event={"ID":"93cd4396-77ee-40b7-be1f-79e58bb2fae5","Type":"ContainerStarted","Data":"5eba44db85bca1de8add08494505960eedd26b79afcf5f666df23a03c7a02a78"} Jun 10 12:54:07.630252 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:07.630128 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" event={"ID":"93cd4396-77ee-40b7-be1f-79e58bb2fae5","Type":"ContainerStarted","Data":"0f6024a3df4ab7694ec98554e9674b6b915b18e7c153bd085a6e5892a2de77bf"} Jun 10 12:54:07.632537 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:07.632507 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45e8ee35-83b9-416a-98a9-45ddfb54437e","Type":"ContainerStarted","Data":"ca20e362a6f2690c1286d0c90f189cd480a9eed70739716efd82e3e39f26c9b8"} Jun 10 12:54:07.634399 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:07.634371 2581 generic.go:356] "Generic (PLEG): container finished" podID="2bdc3c6b-9206-495d-a0a8-142384bfa823" containerID="81fe767197b005c2207e9b74f78c6a1a9b85a17e49f83b93213ad27adce2393f" exitCode=0 Jun 10 12:54:07.634528 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:07.634406 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nqh9k" event={"ID":"2bdc3c6b-9206-495d-a0a8-142384bfa823","Type":"ContainerDied","Data":"81fe767197b005c2207e9b74f78c6a1a9b85a17e49f83b93213ad27adce2393f"} Jun 10 12:54:07.713042 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:07.713018 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:54:08.335776 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:08.335684 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:08.434200 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:08.434136 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:08.496051 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:08.495974 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:08.586458 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:08.586365 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:08.640776 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:08.640742 2581 generic.go:356] "Generic (PLEG): container finished" podID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerID="4bd8f67fea5141907df5cc5db250ed8b78fb73575d57ff1c9dbe78fb4cf9d991" exitCode=0 Jun 10 12:54:08.640929 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:08.640822 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45e8ee35-83b9-416a-98a9-45ddfb54437e","Type":"ContainerDied","Data":"4bd8f67fea5141907df5cc5db250ed8b78fb73575d57ff1c9dbe78fb4cf9d991"} Jun 10 12:54:08.643316 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:08.643294 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nqh9k" event={"ID":"2bdc3c6b-9206-495d-a0a8-142384bfa823","Type":"ContainerStarted","Data":"ab71451e2e95e2cc7bc914e51b68f0113b80d645831b5bec46d216dd65246593"} Jun 10 12:54:08.643439 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:08.643347 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nqh9k" event={"ID":"2bdc3c6b-9206-495d-a0a8-142384bfa823","Type":"ContainerStarted","Data":"ad7011efe2c982c87c162374614f6d651e1ec8d7e083b77f9ed16195df810ce0"} Jun 10 12:54:08.698961 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:08.698897 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nqh9k" podStartSLOduration=2.6023605229999998 podStartE2EDuration="3.698876009s" totalImagesPullingTime="1.012982831s" totalInitContainerRuntime="83.532655ms" isStatefulPod=true podCreationTimestamp="2026-06-10 12:54:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:54:08.697786289 +0000 UTC m=+310.438000783" watchObservedRunningTime="2026-06-10 12:54:08.698876009 +0000 UTC m=+310.439090491" Jun 10 12:54:10.095058 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:10.094977 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:10.164127 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:10.164089 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:10.565854 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:10.565822 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:10.653350 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:10.653324 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d896d57f5-t9t95" event={"ID":"f8d53f03-d7d2-439d-97e4-d19aae54d880","Type":"ContainerStarted","Data":"3b6864ddf7bb4cb5c279f0873c7fb1300f58956fef262b06aa436a1e14c8463e"} Jun 10 12:54:10.655479 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:10.655460 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" event={"ID":"93cd4396-77ee-40b7-be1f-79e58bb2fae5","Type":"ContainerStarted","Data":"f19770a30d6e8d7c090bbb3ef265858a90084d29d39b7b88940e74ce8c83ea26"} Jun 10 12:54:10.679732 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:10.679688 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d896d57f5-t9t95" podStartSLOduration=2.310862685 podStartE2EDuration="5.679673191s" totalImagesPullingTime="3.368810506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:54:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:54:10.678355096 +0000 UTC m=+312.418569596" watchObservedRunningTime="2026-06-10 12:54:10.679673191 +0000 UTC m=+312.419887672" Jun 10 12:54:10.699473 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:10.699426 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-krz75" podStartSLOduration=3.435106267 podStartE2EDuration="5.699410083s" totalImagesPullingTime="2.264303816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:54:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:54:10.697843067 +0000 UTC m=+312.438057561" watchObservedRunningTime="2026-06-10 12:54:10.699410083 +0000 UTC m=+312.439624564" Jun 10 12:54:11.248997 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.248959 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:11.304820 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.304785 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:11.649271 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.649223 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:11.657304 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.657280 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 10 12:54:11.673422 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.673387 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.674628 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.674601 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45e8ee35-83b9-416a-98a9-45ddfb54437e","Type":"ContainerStarted","Data":"d944d83475b0c76b297f046ff6133ae141b071dc39e71240a5c43927d5efc6bd"} Jun 10 12:54:11.674719 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.674658 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45e8ee35-83b9-416a-98a9-45ddfb54437e","Type":"ContainerStarted","Data":"81926b94756d490d9a5e987a104a0be50482e8fb2839eeff21bb59680e4effc3"} Jun 10 12:54:11.674719 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.674684 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45e8ee35-83b9-416a-98a9-45ddfb54437e","Type":"ContainerStarted","Data":"6d131879f5b77ee8527b1a4aaed44ec16963196e181d6f51e5afa0f9995b24aa"} Jun 10 12:54:11.674719 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.674713 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45e8ee35-83b9-416a-98a9-45ddfb54437e","Type":"ContainerStarted","Data":"48b470d8405ce491abe7b7aa4aab7282dec1752b180803a8316b2d968c509acd"} Jun 10 12:54:11.674834 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.674736 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45e8ee35-83b9-416a-98a9-45ddfb54437e","Type":"ContainerStarted","Data":"ac6f96d9f13124f1ac722cc92a6bac12de372ac01a2c00890275f9753d47658b"} Jun 10 12:54:11.676239 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.676219 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 10 12:54:11.676327 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.676222 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 10 12:54:11.678868 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.678845 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5bmmm\"" Jun 10 12:54:11.679070 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.679050 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 10 12:54:11.679297 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.679279 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 10 12:54:11.679444 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.679427 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 10 12:54:11.679551 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.679535 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 10 12:54:11.679619 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.679472 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 10 12:54:11.679677 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.679502 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 10 12:54:11.680252 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.680228 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 10 12:54:11.680720 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.680454 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 10 12:54:11.682236 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.682215 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 10 12:54:11.682625 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.682527 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 10 12:54:11.682820 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.682797 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 10 12:54:11.683410 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.683392 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ek3q6bb37d1oa\"" Jun 10 12:54:11.684068 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.684047 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 10 12:54:11.684298 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.684282 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 10 12:54:11.687013 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.686994 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 10 12:54:11.803880 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.803844 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:11.837000 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.836967 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.837000 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.837004 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7de4b188-af51-4964-a1c7-7eb32bfa7bed-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.837240 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.837022 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.837240 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.837202 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.837240 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.837235 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.837364 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.837283 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-web-config\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.837364 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.837333 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.837468 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.837379 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.837468 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.837401 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.837468 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.837416 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.837468 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.837432 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkmz8\" (UniqueName: \"kubernetes.io/projected/7de4b188-af51-4964-a1c7-7eb32bfa7bed-kube-api-access-rkmz8\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.837651 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.837470 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.837651 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.837509 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.837651 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.837522 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.837804 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.837730 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7de4b188-af51-4964-a1c7-7eb32bfa7bed-config-out\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.837804 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.837759 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.837804 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.837784 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.837947 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.837805 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.838000 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.837977 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.838055 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.838039 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-config\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.907434 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.907330 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:11.938744 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.938707 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.938744 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.938745 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.938995 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.938774 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-web-config\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.938995 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.938798 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.938995 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.938821 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.938995 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.938847 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.938995 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.938866 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.938995 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.938886 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkmz8\" (UniqueName: \"kubernetes.io/projected/7de4b188-af51-4964-a1c7-7eb32bfa7bed-kube-api-access-rkmz8\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.938995 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.938930 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.938995 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.938962 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.938995 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.938984 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.939452 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.939034 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7de4b188-af51-4964-a1c7-7eb32bfa7bed-config-out\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.939452 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.939059 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.939452 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.939084 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.939452 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.939111 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.939452 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.939195 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.939452 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.939258 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-config\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.939452 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.939283 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.939452 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.939314 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7de4b188-af51-4964-a1c7-7eb32bfa7bed-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.939452 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.939339 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.939874 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.939581 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.939874 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.939633 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.940056 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.940016 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.941192 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.940265 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.941192 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.940917 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.942918 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.942890 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.943106 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.943078 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.943339 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.943316 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.943449 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.943427 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.943871 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.943844 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.944073 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.944047 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.944916 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.944863 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-web-config\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.944916 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.944875 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7de4b188-af51-4964-a1c7-7eb32bfa7bed-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.945069 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.944923 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-config\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.945069 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.944981 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.945344 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.945307 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.945426 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.945369 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7de4b188-af51-4964-a1c7-7eb32bfa7bed-config-out\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.946313 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.946296 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.946431 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.946416 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:11.949748 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:11.949721 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkmz8\" (UniqueName: \"kubernetes.io/projected/7de4b188-af51-4964-a1c7-7eb32bfa7bed-kube-api-access-rkmz8\") pod \"prometheus-k8s-0\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:12.009698 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:12.009662 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:12.178915 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:12.178820 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 10 12:54:12.182075 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:54:12.182050 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7de4b188_af51_4964_a1c7_7eb32bfa7bed.slice/crio-cf73d77884cf049eeeac78f8bf7d7cdf5a420c3926e23d9c355c55edf141341d WatchSource:0}: Error finding container cf73d77884cf049eeeac78f8bf7d7cdf5a420c3926e23d9c355c55edf141341d: Status 404 returned error can't find the container with id cf73d77884cf049eeeac78f8bf7d7cdf5a420c3926e23d9c355c55edf141341d Jun 10 12:54:12.679371 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:12.679330 2581 generic.go:356] "Generic (PLEG): container finished" podID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerID="8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933" exitCode=0 Jun 10 12:54:12.679810 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:12.679421 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7de4b188-af51-4964-a1c7-7eb32bfa7bed","Type":"ContainerDied","Data":"8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933"} Jun 10 12:54:12.679810 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:12.679520 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7de4b188-af51-4964-a1c7-7eb32bfa7bed","Type":"ContainerStarted","Data":"cf73d77884cf049eeeac78f8bf7d7cdf5a420c3926e23d9c355c55edf141341d"} Jun 10 12:54:12.683055 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:12.683035 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45e8ee35-83b9-416a-98a9-45ddfb54437e","Type":"ContainerStarted","Data":"96c7f4a498640963ae03094ed3cebc9144e7103d6872b1d9eee9fc5f08483eb4"} Jun 10 12:54:12.748137 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:12.748086 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.9520552589999998 podStartE2EDuration="6.748072491s" totalImagesPullingTime="3.777761417s" totalInitContainerRuntime="18.255815ms" isStatefulPod=false podCreationTimestamp="2026-06-10 12:54:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:54:12.747340074 +0000 UTC m=+314.487554555" watchObservedRunningTime="2026-06-10 12:54:12.748072491 +0000 UTC m=+314.488287020" Jun 10 12:54:15.059641 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:15.059603 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:15.161209 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:15.161154 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:16.187942 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:16.187903 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:16.188334 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:16.187972 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:16.192757 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:16.192737 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:16.321260 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:16.321224 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:16.422183 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:16.422100 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:16.704735 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:16.704695 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7de4b188-af51-4964-a1c7-7eb32bfa7bed","Type":"ContainerStarted","Data":"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb"} Jun 10 12:54:16.704919 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:16.704782 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7de4b188-af51-4964-a1c7-7eb32bfa7bed","Type":"ContainerStarted","Data":"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905"} Jun 10 12:54:16.710324 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:16.710304 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:17.713104 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:17.713071 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7de4b188-af51-4964-a1c7-7eb32bfa7bed","Type":"ContainerStarted","Data":"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da"} Jun 10 12:54:18.256714 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:18.256626 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:18.360694 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:18.360657 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:18.719946 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:18.719902 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7de4b188-af51-4964-a1c7-7eb32bfa7bed","Type":"ContainerStarted","Data":"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68"} Jun 10 12:54:18.720412 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:18.719985 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7de4b188-af51-4964-a1c7-7eb32bfa7bed","Type":"ContainerStarted","Data":"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b"} Jun 10 12:54:18.720412 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:18.720039 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7de4b188-af51-4964-a1c7-7eb32bfa7bed","Type":"ContainerStarted","Data":"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b"} Jun 10 12:54:18.763631 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:18.763572 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.935310936 podStartE2EDuration="7.763548956s" totalImagesPullingTime="4.811077108s" totalInitContainerRuntime="17.160912ms" isStatefulPod=false podCreationTimestamp="2026-06-10 12:54:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:54:18.760718029 +0000 UTC m=+320.500932509" watchObservedRunningTime="2026-06-10 12:54:18.763548956 +0000 UTC m=+320.503763437" Jun 10 12:54:20.675366 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:20.675330 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b777cbcc5-vb962"] Jun 10 12:54:20.793659 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:20.793621 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:20.886187 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:20.886134 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:20.995734 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:20.995649 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:22.010852 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:22.010812 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:54:22.460603 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:22.460554 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:22.524949 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:22.524916 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:24.269871 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:24.269828 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:24.329309 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:24.329277 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:24.779732 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:24.779699 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d896d57f5-t9t95"] Jun 10 12:54:25.260130 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:25.260097 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:25.323578 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:25.323539 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:25.680791 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:25.680749 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:25.757206 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:25.757160 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:40.295187 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:40.295130 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:40.386944 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:40.386913 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:45.695409 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:45.695348 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-b777cbcc5-vb962" podUID="5e7080a1-50fb-441f-989b-c9f2e897f48c" containerName="registry" containerID="cri-o://1681675b0ad8e9f3da1116a9fe18a001357d171a0d66d97bee1406cac11c6f9a" gracePeriod=30 Jun 10 12:54:45.827990 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:45.827956 2581 generic.go:356] "Generic (PLEG): container finished" podID="5e7080a1-50fb-441f-989b-c9f2e897f48c" containerID="1681675b0ad8e9f3da1116a9fe18a001357d171a0d66d97bee1406cac11c6f9a" exitCode=0 Jun 10 12:54:45.828144 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:45.828034 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b777cbcc5-vb962" event={"ID":"5e7080a1-50fb-441f-989b-c9f2e897f48c","Type":"ContainerDied","Data":"1681675b0ad8e9f3da1116a9fe18a001357d171a0d66d97bee1406cac11c6f9a"} Jun 10 12:54:45.949146 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:45.949093 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:54:46.045436 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.045401 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5e7080a1-50fb-441f-989b-c9f2e897f48c-ca-trust-extracted\") pod \"5e7080a1-50fb-441f-989b-c9f2e897f48c\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " Jun 10 12:54:46.045436 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.045438 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j56q6\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-kube-api-access-j56q6\") pod \"5e7080a1-50fb-441f-989b-c9f2e897f48c\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " Jun 10 12:54:46.045666 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.045461 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5e7080a1-50fb-441f-989b-c9f2e897f48c-image-registry-private-configuration\") pod \"5e7080a1-50fb-441f-989b-c9f2e897f48c\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " Jun 10 12:54:46.045666 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.045482 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-certificates\") pod \"5e7080a1-50fb-441f-989b-c9f2e897f48c\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " Jun 10 12:54:46.045666 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.045532 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5e7080a1-50fb-441f-989b-c9f2e897f48c-trusted-ca\") pod \"5e7080a1-50fb-441f-989b-c9f2e897f48c\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " Jun 10 12:54:46.045666 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.045582 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5e7080a1-50fb-441f-989b-c9f2e897f48c-installation-pull-secrets\") pod \"5e7080a1-50fb-441f-989b-c9f2e897f48c\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " Jun 10 12:54:46.045666 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.045621 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls\") pod \"5e7080a1-50fb-441f-989b-c9f2e897f48c\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " Jun 10 12:54:46.045666 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.045654 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-bound-sa-token\") pod \"5e7080a1-50fb-441f-989b-c9f2e897f48c\" (UID: \"5e7080a1-50fb-441f-989b-c9f2e897f48c\") " Jun 10 12:54:46.046315 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.045971 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5e7080a1-50fb-441f-989b-c9f2e897f48c" (UID: "5e7080a1-50fb-441f-989b-c9f2e897f48c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:54:46.046315 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.046274 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e7080a1-50fb-441f-989b-c9f2e897f48c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5e7080a1-50fb-441f-989b-c9f2e897f48c" (UID: "5e7080a1-50fb-441f-989b-c9f2e897f48c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:54:46.048135 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.048102 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5e7080a1-50fb-441f-989b-c9f2e897f48c" (UID: "5e7080a1-50fb-441f-989b-c9f2e897f48c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:54:46.048441 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.048422 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-kube-api-access-j56q6" (OuterVolumeSpecName: "kube-api-access-j56q6") pod "5e7080a1-50fb-441f-989b-c9f2e897f48c" (UID: "5e7080a1-50fb-441f-989b-c9f2e897f48c"). InnerVolumeSpecName "kube-api-access-j56q6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:54:46.048537 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.048439 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e7080a1-50fb-441f-989b-c9f2e897f48c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5e7080a1-50fb-441f-989b-c9f2e897f48c" (UID: "5e7080a1-50fb-441f-989b-c9f2e897f48c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:54:46.048716 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.048699 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5e7080a1-50fb-441f-989b-c9f2e897f48c" (UID: "5e7080a1-50fb-441f-989b-c9f2e897f48c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:54:46.048788 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.048717 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e7080a1-50fb-441f-989b-c9f2e897f48c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5e7080a1-50fb-441f-989b-c9f2e897f48c" (UID: "5e7080a1-50fb-441f-989b-c9f2e897f48c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:54:46.054024 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.053995 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e7080a1-50fb-441f-989b-c9f2e897f48c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5e7080a1-50fb-441f-989b-c9f2e897f48c" (UID: "5e7080a1-50fb-441f-989b-c9f2e897f48c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:54:46.146282 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.146248 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-tls\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:54:46.146282 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.146274 2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-bound-sa-token\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:54:46.146282 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.146282 2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5e7080a1-50fb-441f-989b-c9f2e897f48c-ca-trust-extracted\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:54:46.146282 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.146293 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j56q6\" (UniqueName: \"kubernetes.io/projected/5e7080a1-50fb-441f-989b-c9f2e897f48c-kube-api-access-j56q6\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:54:46.146529 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.146303 2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5e7080a1-50fb-441f-989b-c9f2e897f48c-image-registry-private-configuration\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:54:46.146529 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.146311 2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5e7080a1-50fb-441f-989b-c9f2e897f48c-registry-certificates\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:54:46.146529 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.146320 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5e7080a1-50fb-441f-989b-c9f2e897f48c-trusted-ca\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:54:46.146529 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.146328 2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5e7080a1-50fb-441f-989b-c9f2e897f48c-installation-pull-secrets\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:54:46.257748 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.257642 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6ccc644f47-p5gx4_59eb3b55-c728-4a20-bc60-758e5ffc296f/router/0.log" Jun 10 12:54:46.264981 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.264934 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bxb7s_c4e62b9a-b59d-4df1-a3fb-8185eb2a5472/serve-healthcheck-canary/0.log" Jun 10 12:54:46.833494 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.833462 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b777cbcc5-vb962" event={"ID":"5e7080a1-50fb-441f-989b-c9f2e897f48c","Type":"ContainerDied","Data":"160a248d8857a89e91bc4d6a3404180c5505076ba5dcf794cf339e0aef3cf54d"} Jun 10 12:54:46.833494 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.833494 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b777cbcc5-vb962" Jun 10 12:54:46.834003 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.833586 2581 scope.go:117] "RemoveContainer" containerID="1681675b0ad8e9f3da1116a9fe18a001357d171a0d66d97bee1406cac11c6f9a" Jun 10 12:54:46.862022 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.861999 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b777cbcc5-vb962"] Jun 10 12:54:46.867100 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:46.867079 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-b777cbcc5-vb962"] Jun 10 12:54:48.717356 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:48.717327 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e7080a1-50fb-441f-989b-c9f2e897f48c" path="/var/lib/kubelet/pods/5e7080a1-50fb-441f-989b-c9f2e897f48c/volumes" Jun 10 12:54:49.805341 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:49.805269 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6d896d57f5-t9t95" podUID="f8d53f03-d7d2-439d-97e4-d19aae54d880" containerName="console" containerID="cri-o://3b6864ddf7bb4cb5c279f0873c7fb1300f58956fef262b06aa436a1e14c8463e" gracePeriod=15 Jun 10 12:54:50.060193 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.060104 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:50.183689 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.183653 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f8d53f03-d7d2-439d-97e4-d19aae54d880-console-config\") pod \"f8d53f03-d7d2-439d-97e4-d19aae54d880\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " Jun 10 12:54:50.183860 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.183719 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f8d53f03-d7d2-439d-97e4-d19aae54d880-service-ca\") pod \"f8d53f03-d7d2-439d-97e4-d19aae54d880\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " Jun 10 12:54:50.183860 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.183739 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f8d53f03-d7d2-439d-97e4-d19aae54d880-console-serving-cert\") pod \"f8d53f03-d7d2-439d-97e4-d19aae54d880\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " Jun 10 12:54:50.183860 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.183773 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f8d53f03-d7d2-439d-97e4-d19aae54d880-oauth-serving-cert\") pod \"f8d53f03-d7d2-439d-97e4-d19aae54d880\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " Jun 10 12:54:50.183860 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.183810 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phhjd\" (UniqueName: \"kubernetes.io/projected/f8d53f03-d7d2-439d-97e4-d19aae54d880-kube-api-access-phhjd\") pod \"f8d53f03-d7d2-439d-97e4-d19aae54d880\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " Jun 10 12:54:50.183860 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.183843 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f8d53f03-d7d2-439d-97e4-d19aae54d880-console-oauth-config\") pod \"f8d53f03-d7d2-439d-97e4-d19aae54d880\" (UID: \"f8d53f03-d7d2-439d-97e4-d19aae54d880\") " Jun 10 12:54:50.184228 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.184154 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8d53f03-d7d2-439d-97e4-d19aae54d880-console-config" (OuterVolumeSpecName: "console-config") pod "f8d53f03-d7d2-439d-97e4-d19aae54d880" (UID: "f8d53f03-d7d2-439d-97e4-d19aae54d880"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:54:50.184228 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.184222 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8d53f03-d7d2-439d-97e4-d19aae54d880-service-ca" (OuterVolumeSpecName: "service-ca") pod "f8d53f03-d7d2-439d-97e4-d19aae54d880" (UID: "f8d53f03-d7d2-439d-97e4-d19aae54d880"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:54:50.184392 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.184266 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8d53f03-d7d2-439d-97e4-d19aae54d880-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f8d53f03-d7d2-439d-97e4-d19aae54d880" (UID: "f8d53f03-d7d2-439d-97e4-d19aae54d880"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:54:50.186226 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.186192 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8d53f03-d7d2-439d-97e4-d19aae54d880-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f8d53f03-d7d2-439d-97e4-d19aae54d880" (UID: "f8d53f03-d7d2-439d-97e4-d19aae54d880"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:54:50.186317 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.186244 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8d53f03-d7d2-439d-97e4-d19aae54d880-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f8d53f03-d7d2-439d-97e4-d19aae54d880" (UID: "f8d53f03-d7d2-439d-97e4-d19aae54d880"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:54:50.186317 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.186274 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8d53f03-d7d2-439d-97e4-d19aae54d880-kube-api-access-phhjd" (OuterVolumeSpecName: "kube-api-access-phhjd") pod "f8d53f03-d7d2-439d-97e4-d19aae54d880" (UID: "f8d53f03-d7d2-439d-97e4-d19aae54d880"). InnerVolumeSpecName "kube-api-access-phhjd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:54:50.285480 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.285442 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f8d53f03-d7d2-439d-97e4-d19aae54d880-service-ca\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:54:50.285480 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.285472 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f8d53f03-d7d2-439d-97e4-d19aae54d880-console-serving-cert\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:54:50.285480 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.285485 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f8d53f03-d7d2-439d-97e4-d19aae54d880-oauth-serving-cert\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:54:50.285701 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.285499 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phhjd\" (UniqueName: \"kubernetes.io/projected/f8d53f03-d7d2-439d-97e4-d19aae54d880-kube-api-access-phhjd\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:54:50.285701 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.285511 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f8d53f03-d7d2-439d-97e4-d19aae54d880-console-oauth-config\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:54:50.285701 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.285523 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f8d53f03-d7d2-439d-97e4-d19aae54d880-console-config\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:54:50.852511 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.852476 2581 generic.go:356] "Generic (PLEG): container finished" podID="f8d53f03-d7d2-439d-97e4-d19aae54d880" containerID="3b6864ddf7bb4cb5c279f0873c7fb1300f58956fef262b06aa436a1e14c8463e" exitCode=0 Jun 10 12:54:50.852955 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.852524 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d896d57f5-t9t95" Jun 10 12:54:50.852955 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.852549 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d896d57f5-t9t95" event={"ID":"f8d53f03-d7d2-439d-97e4-d19aae54d880","Type":"ContainerDied","Data":"3b6864ddf7bb4cb5c279f0873c7fb1300f58956fef262b06aa436a1e14c8463e"} Jun 10 12:54:50.852955 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.852600 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d896d57f5-t9t95" event={"ID":"f8d53f03-d7d2-439d-97e4-d19aae54d880","Type":"ContainerDied","Data":"94b1aa8fa5af02941dbea0760f7395fbad46ccfda508c53e54e899c30b003fb0"} Jun 10 12:54:50.852955 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.852626 2581 scope.go:117] "RemoveContainer" containerID="3b6864ddf7bb4cb5c279f0873c7fb1300f58956fef262b06aa436a1e14c8463e" Jun 10 12:54:50.863949 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.863690 2581 scope.go:117] "RemoveContainer" containerID="3b6864ddf7bb4cb5c279f0873c7fb1300f58956fef262b06aa436a1e14c8463e" Jun 10 12:54:50.864133 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:54:50.864100 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b6864ddf7bb4cb5c279f0873c7fb1300f58956fef262b06aa436a1e14c8463e\": container with ID starting with 3b6864ddf7bb4cb5c279f0873c7fb1300f58956fef262b06aa436a1e14c8463e not found: ID does not exist" containerID="3b6864ddf7bb4cb5c279f0873c7fb1300f58956fef262b06aa436a1e14c8463e" Jun 10 12:54:50.864227 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.864144 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b6864ddf7bb4cb5c279f0873c7fb1300f58956fef262b06aa436a1e14c8463e"} err="failed to get container status \"3b6864ddf7bb4cb5c279f0873c7fb1300f58956fef262b06aa436a1e14c8463e\": rpc error: code = NotFound desc = could not find container \"3b6864ddf7bb4cb5c279f0873c7fb1300f58956fef262b06aa436a1e14c8463e\": container with ID starting with 3b6864ddf7bb4cb5c279f0873c7fb1300f58956fef262b06aa436a1e14c8463e not found: ID does not exist" Jun 10 12:54:50.870652 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.870630 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d896d57f5-t9t95"] Jun 10 12:54:50.875312 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:50.875287 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d896d57f5-t9t95"] Jun 10 12:54:52.717355 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:52.717315 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8d53f03-d7d2-439d-97e4-d19aae54d880" path="/var/lib/kubelet/pods/f8d53f03-d7d2-439d-97e4-d19aae54d880/volumes" Jun 10 12:54:57.432123 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:57.432090 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:57.569707 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:57.569669 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:54:57.669360 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:54:57.669322 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:55:03.907223 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:03.907193 2581 generic.go:356] "Generic (PLEG): container finished" podID="a7d867ca-ca36-479d-b023-738ff3414794" containerID="eb2073e430813db286b3d00070a9265c698cdd2abc48e3ddf8a0a06abaa52297" exitCode=0 Jun 10 12:55:03.907223 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:03.907228 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-vw5nk" event={"ID":"a7d867ca-ca36-479d-b023-738ff3414794","Type":"ContainerDied","Data":"eb2073e430813db286b3d00070a9265c698cdd2abc48e3ddf8a0a06abaa52297"} Jun 10 12:55:03.907633 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:03.907527 2581 scope.go:117] "RemoveContainer" containerID="eb2073e430813db286b3d00070a9265c698cdd2abc48e3ddf8a0a06abaa52297" Jun 10 12:55:04.912080 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:04.912044 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-vw5nk" event={"ID":"a7d867ca-ca36-479d-b023-738ff3414794","Type":"ContainerStarted","Data":"e1f0408baeee275f968af1d826c80362158ddad4172a4d9ead1682024cca1b74"} Jun 10 12:55:12.010589 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:12.010552 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:12.026757 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:12.026727 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:12.580683 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:12.580644 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs\") pod \"network-metrics-daemon-2xw2r\" (UID: \"080cea4a-0afb-41b3-a04c-5763fbb54eac\") " pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:55:12.583095 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:12.583076 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/080cea4a-0afb-41b3-a04c-5763fbb54eac-metrics-certs\") pod \"network-metrics-daemon-2xw2r\" (UID: \"080cea4a-0afb-41b3-a04c-5763fbb54eac\") " pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:55:12.816420 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:12.816395 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jmsw2\"" Jun 10 12:55:12.823886 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:12.823869 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xw2r" Jun 10 12:55:12.955794 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:12.955761 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:12.960994 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:12.960969 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2xw2r"] Jun 10 12:55:12.963938 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:55:12.963913 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod080cea4a_0afb_41b3_a04c_5763fbb54eac.slice/crio-d8e7c9b14e08ebc8c7540f8bcd8d5de26ed90fcade63f7b8d65bc1af4f00cc84 WatchSource:0}: Error finding container d8e7c9b14e08ebc8c7540f8bcd8d5de26ed90fcade63f7b8d65bc1af4f00cc84: Status 404 returned error can't find the container with id d8e7c9b14e08ebc8c7540f8bcd8d5de26ed90fcade63f7b8d65bc1af4f00cc84 Jun 10 12:55:13.945581 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:13.945554 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2xw2r" event={"ID":"080cea4a-0afb-41b3-a04c-5763fbb54eac","Type":"ContainerStarted","Data":"71ae942e95be34307e55d380e1e0bb32478da93f8bf0d28e3637283ee14c2406"} Jun 10 12:55:13.945899 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:13.945613 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2xw2r" event={"ID":"080cea4a-0afb-41b3-a04c-5763fbb54eac","Type":"ContainerStarted","Data":"d8e7c9b14e08ebc8c7540f8bcd8d5de26ed90fcade63f7b8d65bc1af4f00cc84"} Jun 10 12:55:14.951017 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:14.950980 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2xw2r" event={"ID":"080cea4a-0afb-41b3-a04c-5763fbb54eac","Type":"ContainerStarted","Data":"803cd44e6713e271885ee01e94025be88ef6879257f7c60ef14d0588f99d89a5"} Jun 10 12:55:14.976453 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:14.976404 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2xw2r" podStartSLOduration=376.097980974 podStartE2EDuration="6m16.976388111s" totalImagesPullingTime="878.407137ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:48:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:55:14.976012419 +0000 UTC m=+376.716226900" watchObservedRunningTime="2026-06-10 12:55:14.976388111 +0000 UTC m=+376.716602646" Jun 10 12:55:15.123706 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:15.123642 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:55:15.222049 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:15.221948 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:55:18.075644 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:18.075614 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:55:18.157486 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:18.157453 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:55:18.249957 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:18.249928 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:55:21.411011 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:21.410947 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:55:21.474540 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:21.474499 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:55:24.721773 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:24.721739 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:55:25.023542 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:25.023452 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:55:25.776418 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:25.776360 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 10 12:55:25.777288 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:25.777254 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="alertmanager" containerID="cri-o://ac6f96d9f13124f1ac722cc92a6bac12de372ac01a2c00890275f9753d47658b" gracePeriod=120 Jun 10 12:55:25.778350 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:25.777286 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="kube-rbac-proxy-metric" containerID="cri-o://d944d83475b0c76b297f046ff6133ae141b071dc39e71240a5c43927d5efc6bd" gracePeriod=120 Jun 10 12:55:25.778350 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:25.777460 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="prom-label-proxy" containerID="cri-o://96c7f4a498640963ae03094ed3cebc9144e7103d6872b1d9eee9fc5f08483eb4" gracePeriod=120 Jun 10 12:55:25.778350 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:25.777511 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="config-reloader" containerID="cri-o://48b470d8405ce491abe7b7aa4aab7282dec1752b180803a8316b2d968c509acd" gracePeriod=120 Jun 10 12:55:25.778350 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:25.777528 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="kube-rbac-proxy" containerID="cri-o://81926b94756d490d9a5e987a104a0be50482e8fb2839eeff21bb59680e4effc3" gracePeriod=120 Jun 10 12:55:25.778350 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:25.777601 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="kube-rbac-proxy-web" containerID="cri-o://6d131879f5b77ee8527b1a4aaed44ec16963196e181d6f51e5afa0f9995b24aa" gracePeriod=120 Jun 10 12:55:25.999842 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:25.999813 2581 generic.go:356] "Generic (PLEG): container finished" podID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerID="96c7f4a498640963ae03094ed3cebc9144e7103d6872b1d9eee9fc5f08483eb4" exitCode=0 Jun 10 12:55:25.999842 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:25.999839 2581 generic.go:356] "Generic (PLEG): container finished" podID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerID="81926b94756d490d9a5e987a104a0be50482e8fb2839eeff21bb59680e4effc3" exitCode=0 Jun 10 12:55:25.999842 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:25.999845 2581 generic.go:356] "Generic (PLEG): container finished" podID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerID="48b470d8405ce491abe7b7aa4aab7282dec1752b180803a8316b2d968c509acd" exitCode=0 Jun 10 12:55:25.999842 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:25.999851 2581 generic.go:356] "Generic (PLEG): container finished" podID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerID="ac6f96d9f13124f1ac722cc92a6bac12de372ac01a2c00890275f9753d47658b" exitCode=0 Jun 10 12:55:26.000089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:25.999882 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45e8ee35-83b9-416a-98a9-45ddfb54437e","Type":"ContainerDied","Data":"96c7f4a498640963ae03094ed3cebc9144e7103d6872b1d9eee9fc5f08483eb4"} Jun 10 12:55:26.000089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:25.999950 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45e8ee35-83b9-416a-98a9-45ddfb54437e","Type":"ContainerDied","Data":"81926b94756d490d9a5e987a104a0be50482e8fb2839eeff21bb59680e4effc3"} Jun 10 12:55:26.000089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:25.999975 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45e8ee35-83b9-416a-98a9-45ddfb54437e","Type":"ContainerDied","Data":"48b470d8405ce491abe7b7aa4aab7282dec1752b180803a8316b2d968c509acd"} Jun 10 12:55:26.000089 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:26.000000 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45e8ee35-83b9-416a-98a9-45ddfb54437e","Type":"ContainerDied","Data":"ac6f96d9f13124f1ac722cc92a6bac12de372ac01a2c00890275f9753d47658b"} Jun 10 12:55:27.008466 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.008438 2581 generic.go:356] "Generic (PLEG): container finished" podID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerID="d944d83475b0c76b297f046ff6133ae141b071dc39e71240a5c43927d5efc6bd" exitCode=0 Jun 10 12:55:27.008466 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.008464 2581 generic.go:356] "Generic (PLEG): container finished" podID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerID="6d131879f5b77ee8527b1a4aaed44ec16963196e181d6f51e5afa0f9995b24aa" exitCode=0 Jun 10 12:55:27.008812 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.008501 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45e8ee35-83b9-416a-98a9-45ddfb54437e","Type":"ContainerDied","Data":"d944d83475b0c76b297f046ff6133ae141b071dc39e71240a5c43927d5efc6bd"} Jun 10 12:55:27.008812 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.008585 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45e8ee35-83b9-416a-98a9-45ddfb54437e","Type":"ContainerDied","Data":"6d131879f5b77ee8527b1a4aaed44ec16963196e181d6f51e5afa0f9995b24aa"} Jun 10 12:55:27.008812 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.008630 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45e8ee35-83b9-416a-98a9-45ddfb54437e","Type":"ContainerDied","Data":"ca20e362a6f2690c1286d0c90f189cd480a9eed70739716efd82e3e39f26c9b8"} Jun 10 12:55:27.008812 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.008663 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca20e362a6f2690c1286d0c90f189cd480a9eed70739716efd82e3e39f26c9b8" Jun 10 12:55:27.027881 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.027857 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:27.105084 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.105000 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45e8ee35-83b9-416a-98a9-45ddfb54437e-metrics-client-ca\") pod \"45e8ee35-83b9-416a-98a9-45ddfb54437e\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " Jun 10 12:55:27.105084 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.105049 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/45e8ee35-83b9-416a-98a9-45ddfb54437e-alertmanager-main-db\") pod \"45e8ee35-83b9-416a-98a9-45ddfb54437e\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " Jun 10 12:55:27.105084 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.105078 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"45e8ee35-83b9-416a-98a9-45ddfb54437e\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " Jun 10 12:55:27.105373 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.105095 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45e8ee35-83b9-416a-98a9-45ddfb54437e-alertmanager-trusted-ca-bundle\") pod \"45e8ee35-83b9-416a-98a9-45ddfb54437e\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " Jun 10 12:55:27.105373 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.105115 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rhjs\" (UniqueName: \"kubernetes.io/projected/45e8ee35-83b9-416a-98a9-45ddfb54437e-kube-api-access-8rhjs\") pod \"45e8ee35-83b9-416a-98a9-45ddfb54437e\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " Jun 10 12:55:27.105373 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.105142 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/45e8ee35-83b9-416a-98a9-45ddfb54437e-config-out\") pod \"45e8ee35-83b9-416a-98a9-45ddfb54437e\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " Jun 10 12:55:27.105373 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.105195 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-cluster-tls-config\") pod \"45e8ee35-83b9-416a-98a9-45ddfb54437e\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " Jun 10 12:55:27.105373 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.105224 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/45e8ee35-83b9-416a-98a9-45ddfb54437e-tls-assets\") pod \"45e8ee35-83b9-416a-98a9-45ddfb54437e\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " Jun 10 12:55:27.105373 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.105256 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-kube-rbac-proxy-web\") pod \"45e8ee35-83b9-416a-98a9-45ddfb54437e\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " Jun 10 12:55:27.105373 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.105281 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-kube-rbac-proxy\") pod \"45e8ee35-83b9-416a-98a9-45ddfb54437e\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " Jun 10 12:55:27.105373 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.105309 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-main-tls\") pod \"45e8ee35-83b9-416a-98a9-45ddfb54437e\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " Jun 10 12:55:27.105373 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.105348 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-config-volume\") pod \"45e8ee35-83b9-416a-98a9-45ddfb54437e\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " Jun 10 12:55:27.105866 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.105399 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-web-config\") pod \"45e8ee35-83b9-416a-98a9-45ddfb54437e\" (UID: \"45e8ee35-83b9-416a-98a9-45ddfb54437e\") " Jun 10 12:55:27.105866 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.105513 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45e8ee35-83b9-416a-98a9-45ddfb54437e-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "45e8ee35-83b9-416a-98a9-45ddfb54437e" (UID: "45e8ee35-83b9-416a-98a9-45ddfb54437e"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:55:27.105866 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.105531 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45e8ee35-83b9-416a-98a9-45ddfb54437e-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "45e8ee35-83b9-416a-98a9-45ddfb54437e" (UID: "45e8ee35-83b9-416a-98a9-45ddfb54437e"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:55:27.105866 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.105572 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45e8ee35-83b9-416a-98a9-45ddfb54437e-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "45e8ee35-83b9-416a-98a9-45ddfb54437e" (UID: "45e8ee35-83b9-416a-98a9-45ddfb54437e"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:55:27.105866 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.105743 2581 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45e8ee35-83b9-416a-98a9-45ddfb54437e-metrics-client-ca\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:27.105866 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.105762 2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/45e8ee35-83b9-416a-98a9-45ddfb54437e-alertmanager-main-db\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:27.105866 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.105778 2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45e8ee35-83b9-416a-98a9-45ddfb54437e-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:27.108517 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.108488 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "45e8ee35-83b9-416a-98a9-45ddfb54437e" (UID: "45e8ee35-83b9-416a-98a9-45ddfb54437e"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:55:27.109128 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.109046 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "45e8ee35-83b9-416a-98a9-45ddfb54437e" (UID: "45e8ee35-83b9-416a-98a9-45ddfb54437e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:55:27.109128 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.109056 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-config-volume" (OuterVolumeSpecName: "config-volume") pod "45e8ee35-83b9-416a-98a9-45ddfb54437e" (UID: "45e8ee35-83b9-416a-98a9-45ddfb54437e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:55:27.109781 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.109746 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "45e8ee35-83b9-416a-98a9-45ddfb54437e" (UID: "45e8ee35-83b9-416a-98a9-45ddfb54437e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:55:27.110445 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.110417 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "45e8ee35-83b9-416a-98a9-45ddfb54437e" (UID: "45e8ee35-83b9-416a-98a9-45ddfb54437e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:55:27.110940 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.110911 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45e8ee35-83b9-416a-98a9-45ddfb54437e-config-out" (OuterVolumeSpecName: "config-out") pod "45e8ee35-83b9-416a-98a9-45ddfb54437e" (UID: "45e8ee35-83b9-416a-98a9-45ddfb54437e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:55:27.110940 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.110918 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45e8ee35-83b9-416a-98a9-45ddfb54437e-kube-api-access-8rhjs" (OuterVolumeSpecName: "kube-api-access-8rhjs") pod "45e8ee35-83b9-416a-98a9-45ddfb54437e" (UID: "45e8ee35-83b9-416a-98a9-45ddfb54437e"). InnerVolumeSpecName "kube-api-access-8rhjs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:55:27.111134 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.111117 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45e8ee35-83b9-416a-98a9-45ddfb54437e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "45e8ee35-83b9-416a-98a9-45ddfb54437e" (UID: "45e8ee35-83b9-416a-98a9-45ddfb54437e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:55:27.113453 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.113427 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "45e8ee35-83b9-416a-98a9-45ddfb54437e" (UID: "45e8ee35-83b9-416a-98a9-45ddfb54437e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:55:27.119799 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.119775 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-web-config" (OuterVolumeSpecName: "web-config") pod "45e8ee35-83b9-416a-98a9-45ddfb54437e" (UID: "45e8ee35-83b9-416a-98a9-45ddfb54437e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:55:27.207128 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.207097 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:27.207128 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.207122 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rhjs\" (UniqueName: \"kubernetes.io/projected/45e8ee35-83b9-416a-98a9-45ddfb54437e-kube-api-access-8rhjs\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:27.207128 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.207132 2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/45e8ee35-83b9-416a-98a9-45ddfb54437e-config-out\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:27.207318 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.207142 2581 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-cluster-tls-config\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:27.207318 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.207151 2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/45e8ee35-83b9-416a-98a9-45ddfb54437e-tls-assets\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:27.207318 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.207159 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:27.207318 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.207186 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:27.207318 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.207196 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-secret-alertmanager-main-tls\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:27.207318 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.207204 2581 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-config-volume\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:27.207318 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:27.207212 2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/45e8ee35-83b9-416a-98a9-45ddfb54437e-web-config\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:28.012860 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.012831 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.038562 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.038532 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 10 12:55:28.043634 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.043610 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 10 12:55:28.078423 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078400 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 10 12:55:28.078663 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078652 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8d53f03-d7d2-439d-97e4-d19aae54d880" containerName="console" Jun 10 12:55:28.078703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078665 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8d53f03-d7d2-439d-97e4-d19aae54d880" containerName="console" Jun 10 12:55:28.078703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078674 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="prom-label-proxy" Jun 10 12:55:28.078703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078680 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="prom-label-proxy" Jun 10 12:55:28.078703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078689 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="alertmanager" Jun 10 12:55:28.078703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078695 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="alertmanager" Jun 10 12:55:28.078703 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078703 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="kube-rbac-proxy-metric" Jun 10 12:55:28.078872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078708 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="kube-rbac-proxy-metric" Jun 10 12:55:28.078872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078717 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e7080a1-50fb-441f-989b-c9f2e897f48c" containerName="registry" Jun 10 12:55:28.078872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078721 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e7080a1-50fb-441f-989b-c9f2e897f48c" containerName="registry" Jun 10 12:55:28.078872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078730 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="kube-rbac-proxy" Jun 10 12:55:28.078872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078734 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="kube-rbac-proxy" Jun 10 12:55:28.078872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078742 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="init-config-reloader" Jun 10 12:55:28.078872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078747 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="init-config-reloader" Jun 10 12:55:28.078872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078753 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="config-reloader" Jun 10 12:55:28.078872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078759 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="config-reloader" Jun 10 12:55:28.078872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078764 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="kube-rbac-proxy-web" Jun 10 12:55:28.078872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078769 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="kube-rbac-proxy-web" Jun 10 12:55:28.078872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078813 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="kube-rbac-proxy" Jun 10 12:55:28.078872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078820 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="alertmanager" Jun 10 12:55:28.078872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078826 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="prom-label-proxy" Jun 10 12:55:28.078872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078832 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="config-reloader" Jun 10 12:55:28.078872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078838 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8d53f03-d7d2-439d-97e4-d19aae54d880" containerName="console" Jun 10 12:55:28.078872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078844 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="kube-rbac-proxy-metric" Jun 10 12:55:28.078872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078849 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e7080a1-50fb-441f-989b-c9f2e897f48c" containerName="registry" Jun 10 12:55:28.078872 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.078854 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" containerName="kube-rbac-proxy-web" Jun 10 12:55:28.083376 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.083360 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.086072 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.086046 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 10 12:55:28.086202 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.086079 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 10 12:55:28.086202 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.086094 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 10 12:55:28.086202 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.086145 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 10 12:55:28.086202 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.086082 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 10 12:55:28.086566 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.086551 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 10 12:55:28.086630 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.086613 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 10 12:55:28.086695 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.086677 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 10 12:55:28.086754 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.086706 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qfwxh\"" Jun 10 12:55:28.091211 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.091114 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 10 12:55:28.096093 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.096072 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 10 12:55:28.215503 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.215469 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-web-config\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.215688 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.215511 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.215688 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.215528 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/62c9fa3e-6551-4381-a49e-01a640e70389-tls-assets\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.215688 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.215546 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/62c9fa3e-6551-4381-a49e-01a640e70389-config-out\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.215688 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.215610 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.215688 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.215654 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.215688 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.215675 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.215885 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.215714 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-config-volume\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.215885 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.215752 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/62c9fa3e-6551-4381-a49e-01a640e70389-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.215885 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.215781 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62c9fa3e-6551-4381-a49e-01a640e70389-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.215885 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.215808 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6crw\" (UniqueName: \"kubernetes.io/projected/62c9fa3e-6551-4381-a49e-01a640e70389-kube-api-access-z6crw\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.215885 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.215827 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62c9fa3e-6551-4381-a49e-01a640e70389-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.215885 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.215880 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.317268 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.317230 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.317374 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.317276 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/62c9fa3e-6551-4381-a49e-01a640e70389-tls-assets\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.317374 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.317298 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/62c9fa3e-6551-4381-a49e-01a640e70389-config-out\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.317374 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.317319 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.317374 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.317347 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.317374 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.317364 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.317613 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.317387 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-config-volume\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.317613 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.317429 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/62c9fa3e-6551-4381-a49e-01a640e70389-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.317613 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.317468 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62c9fa3e-6551-4381-a49e-01a640e70389-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.317613 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.317495 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6crw\" (UniqueName: \"kubernetes.io/projected/62c9fa3e-6551-4381-a49e-01a640e70389-kube-api-access-z6crw\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.317613 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.317520 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62c9fa3e-6551-4381-a49e-01a640e70389-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.317613 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.317546 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.317613 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.317586 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-web-config\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.319443 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.318524 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/62c9fa3e-6551-4381-a49e-01a640e70389-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.319443 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.319151 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62c9fa3e-6551-4381-a49e-01a640e70389-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.320904 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.320583 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.320904 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.320601 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.320904 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.320830 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.320904 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.320847 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.320904 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.320870 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-config-volume\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.321293 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.321273 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62c9fa3e-6551-4381-a49e-01a640e70389-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.321293 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.321282 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-web-config\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.321483 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.321461 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/62c9fa3e-6551-4381-a49e-01a640e70389-tls-assets\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.321707 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.321693 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/62c9fa3e-6551-4381-a49e-01a640e70389-config-out\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.322696 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.322680 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/62c9fa3e-6551-4381-a49e-01a640e70389-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.327527 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.327501 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6crw\" (UniqueName: \"kubernetes.io/projected/62c9fa3e-6551-4381-a49e-01a640e70389-kube-api-access-z6crw\") pod \"alertmanager-main-0\" (UID: \"62c9fa3e-6551-4381-a49e-01a640e70389\") " pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.394414 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.394389 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 10 12:55:28.538229 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.538205 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 10 12:55:28.540195 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:55:28.540146 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62c9fa3e_6551_4381_a49e_01a640e70389.slice/crio-c2d5b860e79b5f2b30c27c3d5d5655cd69767199b8881de01829995327b2572d WatchSource:0}: Error finding container c2d5b860e79b5f2b30c27c3d5d5655cd69767199b8881de01829995327b2572d: Status 404 returned error can't find the container with id c2d5b860e79b5f2b30c27c3d5d5655cd69767199b8881de01829995327b2572d Jun 10 12:55:28.721011 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:28.717618 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45e8ee35-83b9-416a-98a9-45ddfb54437e" path="/var/lib/kubelet/pods/45e8ee35-83b9-416a-98a9-45ddfb54437e/volumes" Jun 10 12:55:29.018676 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:29.018639 2581 generic.go:356] "Generic (PLEG): container finished" podID="62c9fa3e-6551-4381-a49e-01a640e70389" containerID="4af13ade1ee301586f4555d878099df89482180bcd85da2a283d101945a5f85a" exitCode=0 Jun 10 12:55:29.019033 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:29.018727 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"62c9fa3e-6551-4381-a49e-01a640e70389","Type":"ContainerDied","Data":"4af13ade1ee301586f4555d878099df89482180bcd85da2a283d101945a5f85a"} Jun 10 12:55:29.019033 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:29.018794 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"62c9fa3e-6551-4381-a49e-01a640e70389","Type":"ContainerStarted","Data":"c2d5b860e79b5f2b30c27c3d5d5655cd69767199b8881de01829995327b2572d"} Jun 10 12:55:30.026162 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.026127 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"62c9fa3e-6551-4381-a49e-01a640e70389","Type":"ContainerStarted","Data":"dbac6ee725eda6992ada2fc28c0b52ff467c5c0194d1ee57aefcab88fc8b280a"} Jun 10 12:55:30.026560 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.026209 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"62c9fa3e-6551-4381-a49e-01a640e70389","Type":"ContainerStarted","Data":"7f6920ef16d7188f3baa4b947e88b4a2a1e9735f60b761ee2785c63c77a0ab80"} Jun 10 12:55:30.026560 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.026238 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"62c9fa3e-6551-4381-a49e-01a640e70389","Type":"ContainerStarted","Data":"be7646c48f5b973ebbc42af6cf790505e8b32c7af03a33aa31fad33c4f6b07f7"} Jun 10 12:55:30.026560 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.026266 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"62c9fa3e-6551-4381-a49e-01a640e70389","Type":"ContainerStarted","Data":"9ec45b3e31d720f7898b091f18e7e2f956178c19b517055549c379f590ec26a2"} Jun 10 12:55:30.026560 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.026290 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"62c9fa3e-6551-4381-a49e-01a640e70389","Type":"ContainerStarted","Data":"eb58aa58faa5a79e747db08bf4271020f11605646248d888f73ff04649914055"} Jun 10 12:55:30.026560 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.026313 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"62c9fa3e-6551-4381-a49e-01a640e70389","Type":"ContainerStarted","Data":"e99ff1e6363f3389bee7be3cfe5805b779f7d4969a1bd01b2d5b1408509be8df"} Jun 10 12:55:30.061024 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.060968 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.046490944 podStartE2EDuration="2.060949574s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.45863ms" isStatefulPod=false podCreationTimestamp="2026-06-10 12:55:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:55:30.058653376 +0000 UTC m=+391.798867883" watchObservedRunningTime="2026-06-10 12:55:30.060949574 +0000 UTC m=+391.801164055" Jun 10 12:55:30.136898 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.136862 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 10 12:55:30.137486 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.137237 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="prometheus" containerID="cri-o://36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905" gracePeriod=600 Jun 10 12:55:30.137486 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.137280 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="kube-rbac-proxy-thanos" containerID="cri-o://7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68" gracePeriod=600 Jun 10 12:55:30.137486 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.137306 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="config-reloader" containerID="cri-o://c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb" gracePeriod=600 Jun 10 12:55:30.137486 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.137283 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="kube-rbac-proxy-web" containerID="cri-o://9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b" gracePeriod=600 Jun 10 12:55:30.137486 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.137391 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="thanos-sidecar" containerID="cri-o://6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da" gracePeriod=600 Jun 10 12:55:30.138388 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.138041 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="kube-rbac-proxy" containerID="cri-o://ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b" gracePeriod=600 Jun 10 12:55:30.389014 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.388990 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:30.439764 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.439735 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkmz8\" (UniqueName: \"kubernetes.io/projected/7de4b188-af51-4964-a1c7-7eb32bfa7bed-kube-api-access-rkmz8\") pod \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " Jun 10 12:55:30.439764 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.439771 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " Jun 10 12:55:30.440202 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.439788 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-rulefiles-0\") pod \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " Jun 10 12:55:30.440202 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.439809 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7de4b188-af51-4964-a1c7-7eb32bfa7bed-config-out\") pod \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " Jun 10 12:55:30.440202 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.439837 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-configmap-metrics-client-ca\") pod \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " Jun 10 12:55:30.440202 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.439855 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-rulefiles-2\") pod \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " Jun 10 12:55:30.440202 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.439918 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-thanos-prometheus-http-client-file\") pod \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " Jun 10 12:55:30.440202 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.439945 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-db\") pod \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " Jun 10 12:55:30.440202 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.439970 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-configmap-serving-certs-ca-bundle\") pod \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " Jun 10 12:55:30.440202 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.439996 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-config\") pod \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " Jun 10 12:55:30.440202 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.440032 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-kube-rbac-proxy\") pod \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " Jun 10 12:55:30.440202 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.440057 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-rulefiles-1\") pod \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " Jun 10 12:55:30.440202 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.440096 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-prometheus-k8s-tls\") pod \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " Jun 10 12:55:30.440202 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.440136 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-trusted-ca-bundle\") pod \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " Jun 10 12:55:30.440202 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.440162 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-metrics-client-certs\") pod \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " Jun 10 12:55:30.440964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.440218 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " Jun 10 12:55:30.440964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.440262 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-grpc-tls\") pod \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " Jun 10 12:55:30.440964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.440291 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7de4b188-af51-4964-a1c7-7eb32bfa7bed-tls-assets\") pod \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " Jun 10 12:55:30.440964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.440323 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-configmap-kubelet-serving-ca-bundle\") pod \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " Jun 10 12:55:30.440964 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.440354 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-web-config\") pod \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\" (UID: \"7de4b188-af51-4964-a1c7-7eb32bfa7bed\") " Jun 10 12:55:30.441244 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.441056 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "7de4b188-af51-4964-a1c7-7eb32bfa7bed" (UID: "7de4b188-af51-4964-a1c7-7eb32bfa7bed"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:55:30.442816 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.441348 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "7de4b188-af51-4964-a1c7-7eb32bfa7bed" (UID: "7de4b188-af51-4964-a1c7-7eb32bfa7bed"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:55:30.442816 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.441774 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "7de4b188-af51-4964-a1c7-7eb32bfa7bed" (UID: "7de4b188-af51-4964-a1c7-7eb32bfa7bed"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:55:30.442816 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.442334 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "7de4b188-af51-4964-a1c7-7eb32bfa7bed" (UID: "7de4b188-af51-4964-a1c7-7eb32bfa7bed"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:55:30.442816 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.442499 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7de4b188-af51-4964-a1c7-7eb32bfa7bed-kube-api-access-rkmz8" (OuterVolumeSpecName: "kube-api-access-rkmz8") pod "7de4b188-af51-4964-a1c7-7eb32bfa7bed" (UID: "7de4b188-af51-4964-a1c7-7eb32bfa7bed"). InnerVolumeSpecName "kube-api-access-rkmz8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:55:30.442816 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.442769 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de4b188-af51-4964-a1c7-7eb32bfa7bed-config-out" (OuterVolumeSpecName: "config-out") pod "7de4b188-af51-4964-a1c7-7eb32bfa7bed" (UID: "7de4b188-af51-4964-a1c7-7eb32bfa7bed"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 12:55:30.443224 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.442828 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "7de4b188-af51-4964-a1c7-7eb32bfa7bed" (UID: "7de4b188-af51-4964-a1c7-7eb32bfa7bed"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:55:30.443224 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.442966 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "7de4b188-af51-4964-a1c7-7eb32bfa7bed" (UID: "7de4b188-af51-4964-a1c7-7eb32bfa7bed"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:55:30.443542 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.443449 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "7de4b188-af51-4964-a1c7-7eb32bfa7bed" (UID: "7de4b188-af51-4964-a1c7-7eb32bfa7bed"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:55:30.443542 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.443506 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "7de4b188-af51-4964-a1c7-7eb32bfa7bed" (UID: "7de4b188-af51-4964-a1c7-7eb32bfa7bed"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 12:55:30.445071 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.445046 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-config" (OuterVolumeSpecName: "config") pod "7de4b188-af51-4964-a1c7-7eb32bfa7bed" (UID: "7de4b188-af51-4964-a1c7-7eb32bfa7bed"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:55:30.445953 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.445896 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "7de4b188-af51-4964-a1c7-7eb32bfa7bed" (UID: "7de4b188-af51-4964-a1c7-7eb32bfa7bed"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:55:30.449698 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.446828 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "7de4b188-af51-4964-a1c7-7eb32bfa7bed" (UID: "7de4b188-af51-4964-a1c7-7eb32bfa7bed"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:55:30.449698 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.447228 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "7de4b188-af51-4964-a1c7-7eb32bfa7bed" (UID: "7de4b188-af51-4964-a1c7-7eb32bfa7bed"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:55:30.449698 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.448620 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7de4b188-af51-4964-a1c7-7eb32bfa7bed-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7de4b188-af51-4964-a1c7-7eb32bfa7bed" (UID: "7de4b188-af51-4964-a1c7-7eb32bfa7bed"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:55:30.449698 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.448710 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "7de4b188-af51-4964-a1c7-7eb32bfa7bed" (UID: "7de4b188-af51-4964-a1c7-7eb32bfa7bed"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:55:30.449698 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.448747 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "7de4b188-af51-4964-a1c7-7eb32bfa7bed" (UID: "7de4b188-af51-4964-a1c7-7eb32bfa7bed"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:55:30.449698 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.449486 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "7de4b188-af51-4964-a1c7-7eb32bfa7bed" (UID: "7de4b188-af51-4964-a1c7-7eb32bfa7bed"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:55:30.449698 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.449665 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "7de4b188-af51-4964-a1c7-7eb32bfa7bed" (UID: "7de4b188-af51-4964-a1c7-7eb32bfa7bed"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:55:30.458380 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.458357 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-web-config" (OuterVolumeSpecName: "web-config") pod "7de4b188-af51-4964-a1c7-7eb32bfa7bed" (UID: "7de4b188-af51-4964-a1c7-7eb32bfa7bed"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 12:55:30.541771 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.541727 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkmz8\" (UniqueName: \"kubernetes.io/projected/7de4b188-af51-4964-a1c7-7eb32bfa7bed-kube-api-access-rkmz8\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:30.541771 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.541765 2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:30.541771 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.541777 2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:30.541771 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.541787 2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7de4b188-af51-4964-a1c7-7eb32bfa7bed-config-out\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:30.542041 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.541795 2581 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-configmap-metrics-client-ca\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:30.542041 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.541804 2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:30.542041 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.541814 2581 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:30.542041 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.541822 2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-db\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:30.542041 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.541830 2581 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:30.542041 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.541839 2581 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-config\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:30.542041 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.541847 2581 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-kube-rbac-proxy\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:30.542041 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.541856 2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:30.542041 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.541864 2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:30.542041 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.541871 2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:30.542041 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.541879 2581 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-metrics-client-certs\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:30.542041 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.541889 2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:30.542041 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.541897 2581 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-secret-grpc-tls\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:30.542041 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.541905 2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7de4b188-af51-4964-a1c7-7eb32bfa7bed-tls-assets\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:30.542041 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.541912 2581 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7de4b188-af51-4964-a1c7-7eb32bfa7bed-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:30.542041 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:30.541919 2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7de4b188-af51-4964-a1c7-7eb32bfa7bed-web-config\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:55:31.035746 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.035661 2581 generic.go:356] "Generic (PLEG): container finished" podID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerID="7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68" exitCode=0 Jun 10 12:55:31.035746 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.035695 2581 generic.go:356] "Generic (PLEG): container finished" podID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerID="ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b" exitCode=0 Jun 10 12:55:31.035746 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.035704 2581 generic.go:356] "Generic (PLEG): container finished" podID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerID="9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b" exitCode=0 Jun 10 12:55:31.035746 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.035711 2581 generic.go:356] "Generic (PLEG): container finished" podID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerID="6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da" exitCode=0 Jun 10 12:55:31.035746 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.035710 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7de4b188-af51-4964-a1c7-7eb32bfa7bed","Type":"ContainerDied","Data":"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68"} Jun 10 12:55:31.035746 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.035742 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.036346 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.035719 2581 generic.go:356] "Generic (PLEG): container finished" podID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerID="c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb" exitCode=0 Jun 10 12:55:31.036346 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.035803 2581 generic.go:356] "Generic (PLEG): container finished" podID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerID="36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905" exitCode=0 Jun 10 12:55:31.036346 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.035817 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7de4b188-af51-4964-a1c7-7eb32bfa7bed","Type":"ContainerDied","Data":"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b"} Jun 10 12:55:31.036346 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.035858 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7de4b188-af51-4964-a1c7-7eb32bfa7bed","Type":"ContainerDied","Data":"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b"} Jun 10 12:55:31.036346 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.035883 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7de4b188-af51-4964-a1c7-7eb32bfa7bed","Type":"ContainerDied","Data":"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da"} Jun 10 12:55:31.036346 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.035907 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7de4b188-af51-4964-a1c7-7eb32bfa7bed","Type":"ContainerDied","Data":"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb"} Jun 10 12:55:31.036346 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.035930 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7de4b188-af51-4964-a1c7-7eb32bfa7bed","Type":"ContainerDied","Data":"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905"} Jun 10 12:55:31.036346 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.035953 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7de4b188-af51-4964-a1c7-7eb32bfa7bed","Type":"ContainerDied","Data":"cf73d77884cf049eeeac78f8bf7d7cdf5a420c3926e23d9c355c55edf141341d"} Jun 10 12:55:31.037372 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.035863 2581 scope.go:117] "RemoveContainer" containerID="7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68" Jun 10 12:55:31.045221 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.045192 2581 scope.go:117] "RemoveContainer" containerID="ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b" Jun 10 12:55:31.052768 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.052752 2581 scope.go:117] "RemoveContainer" containerID="9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b" Jun 10 12:55:31.059510 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.059490 2581 scope.go:117] "RemoveContainer" containerID="6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da" Jun 10 12:55:31.061263 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.061243 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 10 12:55:31.066806 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.066780 2581 scope.go:117] "RemoveContainer" containerID="c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb" Jun 10 12:55:31.068781 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.068763 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 10 12:55:31.073533 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.073512 2581 scope.go:117] "RemoveContainer" containerID="36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905" Jun 10 12:55:31.080536 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.080521 2581 scope.go:117] "RemoveContainer" containerID="8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933" Jun 10 12:55:31.087400 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.087380 2581 scope.go:117] "RemoveContainer" containerID="7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68" Jun 10 12:55:31.087664 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:55:31.087645 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68\": container with ID starting with 7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68 not found: ID does not exist" containerID="7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68" Jun 10 12:55:31.087733 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.087696 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68"} err="failed to get container status \"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68\": rpc error: code = NotFound desc = could not find container \"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68\": container with ID starting with 7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68 not found: ID does not exist" Jun 10 12:55:31.087733 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.087722 2581 scope.go:117] "RemoveContainer" containerID="ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b" Jun 10 12:55:31.087946 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:55:31.087930 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b\": container with ID starting with ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b not found: ID does not exist" containerID="ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b" Jun 10 12:55:31.088011 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.087954 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b"} err="failed to get container status \"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b\": rpc error: code = NotFound desc = could not find container \"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b\": container with ID starting with ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b not found: ID does not exist" Jun 10 12:55:31.088011 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.087975 2581 scope.go:117] "RemoveContainer" containerID="9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b" Jun 10 12:55:31.088234 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:55:31.088219 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b\": container with ID starting with 9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b not found: ID does not exist" containerID="9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b" Jun 10 12:55:31.088309 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.088239 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b"} err="failed to get container status \"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b\": rpc error: code = NotFound desc = could not find container \"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b\": container with ID starting with 9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b not found: ID does not exist" Jun 10 12:55:31.088309 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.088257 2581 scope.go:117] "RemoveContainer" containerID="6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da" Jun 10 12:55:31.088522 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:55:31.088505 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da\": container with ID starting with 6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da not found: ID does not exist" containerID="6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da" Jun 10 12:55:31.088564 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.088528 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da"} err="failed to get container status \"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da\": rpc error: code = NotFound desc = could not find container \"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da\": container with ID starting with 6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da not found: ID does not exist" Jun 10 12:55:31.088564 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.088550 2581 scope.go:117] "RemoveContainer" containerID="c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb" Jun 10 12:55:31.088781 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:55:31.088766 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb\": container with ID starting with c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb not found: ID does not exist" containerID="c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb" Jun 10 12:55:31.088832 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.088784 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb"} err="failed to get container status \"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb\": rpc error: code = NotFound desc = could not find container \"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb\": container with ID starting with c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb not found: ID does not exist" Jun 10 12:55:31.088832 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.088807 2581 scope.go:117] "RemoveContainer" containerID="36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905" Jun 10 12:55:31.089015 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:55:31.089001 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905\": container with ID starting with 36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905 not found: ID does not exist" containerID="36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905" Jun 10 12:55:31.089051 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.089017 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905"} err="failed to get container status \"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905\": rpc error: code = NotFound desc = could not find container \"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905\": container with ID starting with 36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905 not found: ID does not exist" Jun 10 12:55:31.089051 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.089028 2581 scope.go:117] "RemoveContainer" containerID="8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933" Jun 10 12:55:31.089282 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:55:31.089256 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933\": container with ID starting with 8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933 not found: ID does not exist" containerID="8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933" Jun 10 12:55:31.089374 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.089290 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933"} err="failed to get container status \"8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933\": rpc error: code = NotFound desc = could not find container \"8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933\": container with ID starting with 8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933 not found: ID does not exist" Jun 10 12:55:31.089374 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.089311 2581 scope.go:117] "RemoveContainer" containerID="7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68" Jun 10 12:55:31.089597 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.089579 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68"} err="failed to get container status \"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68\": rpc error: code = NotFound desc = could not find container \"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68\": container with ID starting with 7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68 not found: ID does not exist" Jun 10 12:55:31.089647 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.089608 2581 scope.go:117] "RemoveContainer" containerID="ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b" Jun 10 12:55:31.089810 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.089792 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b"} err="failed to get container status \"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b\": rpc error: code = NotFound desc = could not find container \"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b\": container with ID starting with ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b not found: ID does not exist" Jun 10 12:55:31.089875 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.089811 2581 scope.go:117] "RemoveContainer" containerID="9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b" Jun 10 12:55:31.090058 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.090038 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b"} err="failed to get container status \"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b\": rpc error: code = NotFound desc = could not find container \"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b\": container with ID starting with 9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b not found: ID does not exist" Jun 10 12:55:31.090099 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.090058 2581 scope.go:117] "RemoveContainer" containerID="6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da" Jun 10 12:55:31.090310 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.090287 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da"} err="failed to get container status \"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da\": rpc error: code = NotFound desc = could not find container \"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da\": container with ID starting with 6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da not found: ID does not exist" Jun 10 12:55:31.090362 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.090311 2581 scope.go:117] "RemoveContainer" containerID="c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb" Jun 10 12:55:31.090560 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.090544 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb"} err="failed to get container status \"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb\": rpc error: code = NotFound desc = could not find container \"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb\": container with ID starting with c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb not found: ID does not exist" Jun 10 12:55:31.090612 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.090560 2581 scope.go:117] "RemoveContainer" containerID="36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905" Jun 10 12:55:31.090758 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.090741 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905"} err="failed to get container status \"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905\": rpc error: code = NotFound desc = could not find container \"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905\": container with ID starting with 36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905 not found: ID does not exist" Jun 10 12:55:31.090799 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.090760 2581 scope.go:117] "RemoveContainer" containerID="8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933" Jun 10 12:55:31.090979 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.090964 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933"} err="failed to get container status \"8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933\": rpc error: code = NotFound desc = could not find container \"8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933\": container with ID starting with 8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933 not found: ID does not exist" Jun 10 12:55:31.091028 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.090980 2581 scope.go:117] "RemoveContainer" containerID="7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68" Jun 10 12:55:31.091207 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.091189 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68"} err="failed to get container status \"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68\": rpc error: code = NotFound desc = could not find container \"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68\": container with ID starting with 7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68 not found: ID does not exist" Jun 10 12:55:31.091252 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.091207 2581 scope.go:117] "RemoveContainer" containerID="ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b" Jun 10 12:55:31.091435 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.091418 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b"} err="failed to get container status \"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b\": rpc error: code = NotFound desc = could not find container \"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b\": container with ID starting with ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b not found: ID does not exist" Jun 10 12:55:31.091472 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.091436 2581 scope.go:117] "RemoveContainer" containerID="9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b" Jun 10 12:55:31.091658 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.091643 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b"} err="failed to get container status \"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b\": rpc error: code = NotFound desc = could not find container \"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b\": container with ID starting with 9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b not found: ID does not exist" Jun 10 12:55:31.091702 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.091658 2581 scope.go:117] "RemoveContainer" containerID="6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da" Jun 10 12:55:31.091849 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.091836 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da"} err="failed to get container status \"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da\": rpc error: code = NotFound desc = could not find container \"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da\": container with ID starting with 6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da not found: ID does not exist" Jun 10 12:55:31.091894 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.091849 2581 scope.go:117] "RemoveContainer" containerID="c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb" Jun 10 12:55:31.092059 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.092040 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb"} err="failed to get container status \"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb\": rpc error: code = NotFound desc = could not find container \"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb\": container with ID starting with c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb not found: ID does not exist" Jun 10 12:55:31.092125 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.092070 2581 scope.go:117] "RemoveContainer" containerID="36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905" Jun 10 12:55:31.092304 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.092287 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905"} err="failed to get container status \"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905\": rpc error: code = NotFound desc = could not find container \"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905\": container with ID starting with 36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905 not found: ID does not exist" Jun 10 12:55:31.092348 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.092304 2581 scope.go:117] "RemoveContainer" containerID="8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933" Jun 10 12:55:31.092503 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.092488 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933"} err="failed to get container status \"8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933\": rpc error: code = NotFound desc = could not find container \"8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933\": container with ID starting with 8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933 not found: ID does not exist" Jun 10 12:55:31.092542 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.092503 2581 scope.go:117] "RemoveContainer" containerID="7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68" Jun 10 12:55:31.092710 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.092691 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68"} err="failed to get container status \"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68\": rpc error: code = NotFound desc = could not find container \"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68\": container with ID starting with 7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68 not found: ID does not exist" Jun 10 12:55:31.092773 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.092712 2581 scope.go:117] "RemoveContainer" containerID="ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b" Jun 10 12:55:31.092922 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.092907 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b"} err="failed to get container status \"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b\": rpc error: code = NotFound desc = could not find container \"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b\": container with ID starting with ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b not found: ID does not exist" Jun 10 12:55:31.092972 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.092923 2581 scope.go:117] "RemoveContainer" containerID="9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b" Jun 10 12:55:31.093125 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.093106 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b"} err="failed to get container status \"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b\": rpc error: code = NotFound desc = could not find container \"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b\": container with ID starting with 9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b not found: ID does not exist" Jun 10 12:55:31.093194 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.093126 2581 scope.go:117] "RemoveContainer" containerID="6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da" Jun 10 12:55:31.093353 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.093338 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da"} err="failed to get container status \"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da\": rpc error: code = NotFound desc = could not find container \"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da\": container with ID starting with 6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da not found: ID does not exist" Jun 10 12:55:31.093401 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.093352 2581 scope.go:117] "RemoveContainer" containerID="c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb" Jun 10 12:55:31.093562 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.093545 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb"} err="failed to get container status \"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb\": rpc error: code = NotFound desc = could not find container \"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb\": container with ID starting with c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb not found: ID does not exist" Jun 10 12:55:31.093645 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.093562 2581 scope.go:117] "RemoveContainer" containerID="36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905" Jun 10 12:55:31.093776 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.093757 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905"} err="failed to get container status \"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905\": rpc error: code = NotFound desc = could not find container \"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905\": container with ID starting with 36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905 not found: ID does not exist" Jun 10 12:55:31.093842 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.093788 2581 scope.go:117] "RemoveContainer" containerID="8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933" Jun 10 12:55:31.094021 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.094005 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933"} err="failed to get container status \"8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933\": rpc error: code = NotFound desc = could not find container \"8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933\": container with ID starting with 8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933 not found: ID does not exist" Jun 10 12:55:31.094080 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.094023 2581 scope.go:117] "RemoveContainer" containerID="7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68" Jun 10 12:55:31.094281 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.094261 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68"} err="failed to get container status \"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68\": rpc error: code = NotFound desc = could not find container \"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68\": container with ID starting with 7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68 not found: ID does not exist" Jun 10 12:55:31.094281 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.094281 2581 scope.go:117] "RemoveContainer" containerID="ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b" Jun 10 12:55:31.094516 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.094498 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b"} err="failed to get container status \"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b\": rpc error: code = NotFound desc = could not find container \"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b\": container with ID starting with ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b not found: ID does not exist" Jun 10 12:55:31.094574 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.094517 2581 scope.go:117] "RemoveContainer" containerID="9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b" Jun 10 12:55:31.094740 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.094723 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b"} err="failed to get container status \"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b\": rpc error: code = NotFound desc = could not find container \"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b\": container with ID starting with 9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b not found: ID does not exist" Jun 10 12:55:31.094785 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.094741 2581 scope.go:117] "RemoveContainer" containerID="6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da" Jun 10 12:55:31.094935 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.094921 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da"} err="failed to get container status \"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da\": rpc error: code = NotFound desc = could not find container \"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da\": container with ID starting with 6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da not found: ID does not exist" Jun 10 12:55:31.094990 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.094936 2581 scope.go:117] "RemoveContainer" containerID="c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb" Jun 10 12:55:31.095126 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.095109 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb"} err="failed to get container status \"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb\": rpc error: code = NotFound desc = could not find container \"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb\": container with ID starting with c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb not found: ID does not exist" Jun 10 12:55:31.095202 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.095127 2581 scope.go:117] "RemoveContainer" containerID="36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905" Jun 10 12:55:31.095339 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.095325 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905"} err="failed to get container status \"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905\": rpc error: code = NotFound desc = could not find container \"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905\": container with ID starting with 36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905 not found: ID does not exist" Jun 10 12:55:31.095393 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.095339 2581 scope.go:117] "RemoveContainer" containerID="8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933" Jun 10 12:55:31.095782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.095755 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933"} err="failed to get container status \"8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933\": rpc error: code = NotFound desc = could not find container \"8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933\": container with ID starting with 8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933 not found: ID does not exist" Jun 10 12:55:31.095782 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.095782 2581 scope.go:117] "RemoveContainer" containerID="7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68" Jun 10 12:55:31.096092 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.096067 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68"} err="failed to get container status \"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68\": rpc error: code = NotFound desc = could not find container \"7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68\": container with ID starting with 7798714e4c88571ba2bdcff498ec2daeaef5ffe9c3d525570e4a6a1604898f68 not found: ID does not exist" Jun 10 12:55:31.096206 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.096092 2581 scope.go:117] "RemoveContainer" containerID="ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b" Jun 10 12:55:31.096469 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.096446 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b"} err="failed to get container status \"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b\": rpc error: code = NotFound desc = could not find container \"ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b\": container with ID starting with ff1519777118b8c09e5aed664d79faf8f8f90331d629c373b6a3e930f8b5a87b not found: ID does not exist" Jun 10 12:55:31.096469 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.096469 2581 scope.go:117] "RemoveContainer" containerID="9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b" Jun 10 12:55:31.096728 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.096710 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b"} err="failed to get container status \"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b\": rpc error: code = NotFound desc = could not find container \"9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b\": container with ID starting with 9715faeedb57e6efc70a687cbab2e8fc7c0d0d3fe3367627a15f8806b05c618b not found: ID does not exist" Jun 10 12:55:31.096728 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.096729 2581 scope.go:117] "RemoveContainer" containerID="6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da" Jun 10 12:55:31.097046 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.097024 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da"} err="failed to get container status \"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da\": rpc error: code = NotFound desc = could not find container \"6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da\": container with ID starting with 6ecfbee15b2449c84aa30383c12171d44e137b6b2109a7f1689ee9fe8708f2da not found: ID does not exist" Jun 10 12:55:31.097046 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.097044 2581 scope.go:117] "RemoveContainer" containerID="c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb" Jun 10 12:55:31.097351 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.097323 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb"} err="failed to get container status \"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb\": rpc error: code = NotFound desc = could not find container \"c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb\": container with ID starting with c649a71da3acb341296d698fb1794aac8bd2aa6f33618c3e9469ef5c2b7bcebb not found: ID does not exist" Jun 10 12:55:31.097351 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.097349 2581 scope.go:117] "RemoveContainer" containerID="36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905" Jun 10 12:55:31.097610 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.097591 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905"} err="failed to get container status \"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905\": rpc error: code = NotFound desc = could not find container \"36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905\": container with ID starting with 36990e9bbdecf173885465b75e0810b875607689972d684d953670efded52905 not found: ID does not exist" Jun 10 12:55:31.097610 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.097609 2581 scope.go:117] "RemoveContainer" containerID="8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933" Jun 10 12:55:31.097749 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.097686 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 10 12:55:31.097849 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.097822 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933"} err="failed to get container status \"8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933\": rpc error: code = NotFound desc = could not find container \"8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933\": container with ID starting with 8817ec9e6881ee33f97320a43e1746df26f365490607dad2e0b00aaba5525933 not found: ID does not exist" Jun 10 12:55:31.097984 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.097972 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="init-config-reloader" Jun 10 12:55:31.098020 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.097987 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="init-config-reloader" Jun 10 12:55:31.098020 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.098000 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="thanos-sidecar" Jun 10 12:55:31.098020 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.098009 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="thanos-sidecar" Jun 10 12:55:31.098020 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.098019 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="kube-rbac-proxy-web" Jun 10 12:55:31.098131 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.098026 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="kube-rbac-proxy-web" Jun 10 12:55:31.098131 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.098035 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="prometheus" Jun 10 12:55:31.098131 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.098040 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="prometheus" Jun 10 12:55:31.098131 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.098047 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="config-reloader" Jun 10 12:55:31.098131 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.098052 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="config-reloader" Jun 10 12:55:31.098131 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.098058 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="kube-rbac-proxy" Jun 10 12:55:31.098131 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.098062 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="kube-rbac-proxy" Jun 10 12:55:31.098131 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.098073 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="kube-rbac-proxy-thanos" Jun 10 12:55:31.098131 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.098079 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="kube-rbac-proxy-thanos" Jun 10 12:55:31.098131 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.098127 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="config-reloader" Jun 10 12:55:31.098131 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.098133 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="thanos-sidecar" Jun 10 12:55:31.098481 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.098139 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="kube-rbac-proxy-web" Jun 10 12:55:31.098481 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.098148 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="kube-rbac-proxy" Jun 10 12:55:31.098481 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.098156 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="kube-rbac-proxy-thanos" Jun 10 12:55:31.098481 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.098182 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" containerName="prometheus" Jun 10 12:55:31.103835 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.103819 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.106793 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.106775 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 10 12:55:31.106886 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.106832 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 10 12:55:31.106943 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.106919 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ek3q6bb37d1oa\"" Jun 10 12:55:31.107107 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.107085 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 10 12:55:31.107107 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.107100 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 10 12:55:31.107297 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.107135 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 10 12:55:31.107297 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.107186 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 10 12:55:31.107430 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.107412 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 10 12:55:31.107555 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.107539 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 10 12:55:31.107885 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.107867 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5bmmm\"" Jun 10 12:55:31.107977 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.107887 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 10 12:55:31.107977 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.107869 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 10 12:55:31.108136 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.108015 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 10 12:55:31.108136 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.108025 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 10 12:55:31.108516 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.108493 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 10 12:55:31.110427 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.110403 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 10 12:55:31.113916 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.113895 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 10 12:55:31.122304 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.122286 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 10 12:55:31.247747 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.247711 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f9dfbb49-dd88-409a-b7a9-34f2fec68073-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.247747 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.247743 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.247923 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.247778 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.247923 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.247800 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.247923 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.247843 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.247923 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.247870 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-web-config\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.247923 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.247894 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.247923 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.247916 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f9dfbb49-dd88-409a-b7a9-34f2fec68073-config-out\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.248114 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.247941 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.248114 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.247959 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.248114 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.247975 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.248114 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.248004 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.248114 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.248023 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8n6c\" (UniqueName: \"kubernetes.io/projected/f9dfbb49-dd88-409a-b7a9-34f2fec68073-kube-api-access-s8n6c\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.248114 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.248036 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.248114 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.248055 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.248334 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.248116 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.248334 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.248149 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.248334 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.248190 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-config\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.248334 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.248215 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f9dfbb49-dd88-409a-b7a9-34f2fec68073-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.248334 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.248255 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.349588 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.349501 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f9dfbb49-dd88-409a-b7a9-34f2fec68073-config-out\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.349588 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.349540 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.349588 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.349558 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.349588 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.349573 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.349896 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.349622 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.349896 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.349653 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8n6c\" (UniqueName: \"kubernetes.io/projected/f9dfbb49-dd88-409a-b7a9-34f2fec68073-kube-api-access-s8n6c\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.349896 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.349671 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.349896 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.349699 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.349896 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.349730 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.349896 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.349761 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.349896 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.349784 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-config\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.349896 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.349808 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f9dfbb49-dd88-409a-b7a9-34f2fec68073-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.349896 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.349837 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.349896 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.349881 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f9dfbb49-dd88-409a-b7a9-34f2fec68073-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.350387 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.349903 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.350387 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.349952 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.350387 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.349979 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.350387 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.350018 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.350387 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.350040 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-web-config\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.350387 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.350074 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.350387 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.350263 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.350724 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.350418 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.350724 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.350718 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f9dfbb49-dd88-409a-b7a9-34f2fec68073-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.350830 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.350718 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.350830 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.350803 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.353427 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.351125 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.353832 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.353606 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-config\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.353832 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.353608 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f9dfbb49-dd88-409a-b7a9-34f2fec68073-config-out\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.353832 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.353813 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.354076 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.353826 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.354076 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.353854 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.354076 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.354014 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.354326 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.354268 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.354623 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.354605 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.355079 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.355057 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f9dfbb49-dd88-409a-b7a9-34f2fec68073-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.355797 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.355773 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.356078 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.356058 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-web-config\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.356689 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.356665 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f9dfbb49-dd88-409a-b7a9-34f2fec68073-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.356689 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.356680 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f9dfbb49-dd88-409a-b7a9-34f2fec68073-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.360091 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.360067 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8n6c\" (UniqueName: \"kubernetes.io/projected/f9dfbb49-dd88-409a-b7a9-34f2fec68073-kube-api-access-s8n6c\") pod \"prometheus-k8s-0\" (UID: \"f9dfbb49-dd88-409a-b7a9-34f2fec68073\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.415000 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.414966 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:31.558719 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:31.558688 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 10 12:55:31.561332 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:55:31.561307 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9dfbb49_dd88_409a_b7a9_34f2fec68073.slice/crio-9895f3441a5fd9abed303169ae48f1b2244f415545216dda13a43e7b9de35e2e WatchSource:0}: Error finding container 9895f3441a5fd9abed303169ae48f1b2244f415545216dda13a43e7b9de35e2e: Status 404 returned error can't find the container with id 9895f3441a5fd9abed303169ae48f1b2244f415545216dda13a43e7b9de35e2e Jun 10 12:55:32.040409 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:32.040380 2581 generic.go:356] "Generic (PLEG): container finished" podID="f9dfbb49-dd88-409a-b7a9-34f2fec68073" containerID="605ccec1113f96a34ab185ab784006c6abff3d313193fe74fca91bcdb129b086" exitCode=0 Jun 10 12:55:32.040809 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:32.040481 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f9dfbb49-dd88-409a-b7a9-34f2fec68073","Type":"ContainerDied","Data":"605ccec1113f96a34ab185ab784006c6abff3d313193fe74fca91bcdb129b086"} Jun 10 12:55:32.040809 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:32.040553 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f9dfbb49-dd88-409a-b7a9-34f2fec68073","Type":"ContainerStarted","Data":"9895f3441a5fd9abed303169ae48f1b2244f415545216dda13a43e7b9de35e2e"} Jun 10 12:55:32.717333 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:32.717297 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7de4b188-af51-4964-a1c7-7eb32bfa7bed" path="/var/lib/kubelet/pods/7de4b188-af51-4964-a1c7-7eb32bfa7bed/volumes" Jun 10 12:55:33.048926 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:33.048844 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f9dfbb49-dd88-409a-b7a9-34f2fec68073","Type":"ContainerStarted","Data":"3799cf7c9e1a1b36be5df98fcd958f5ac2c375c6614c249ac2b4d0dbc12b07c9"} Jun 10 12:55:33.048926 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:33.048907 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f9dfbb49-dd88-409a-b7a9-34f2fec68073","Type":"ContainerStarted","Data":"21e8d6dd4b86b9805901ca8caeffa6c57c1ad19e2e08867f6619e4607fee5638"} Jun 10 12:55:33.049316 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:33.048934 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f9dfbb49-dd88-409a-b7a9-34f2fec68073","Type":"ContainerStarted","Data":"01857f0bf4e0fc7ce8b4ce562df7f55a76d005b97c5da69ef407be330c332290"} Jun 10 12:55:33.049316 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:33.048960 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f9dfbb49-dd88-409a-b7a9-34f2fec68073","Type":"ContainerStarted","Data":"78fa7f5836db0c66c21f5e4628c50674c1ef06e3f3029b857401f347386606d6"} Jun 10 12:55:33.049316 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:33.048988 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f9dfbb49-dd88-409a-b7a9-34f2fec68073","Type":"ContainerStarted","Data":"34cee0d23684e205f86f4100bdb6f0e9563d2492163fba2d8a55df792af05c1e"} Jun 10 12:55:33.049316 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:33.049012 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f9dfbb49-dd88-409a-b7a9-34f2fec68073","Type":"ContainerStarted","Data":"3a7cd53aaf234725dae685539851f8a163638a24108ecbbc4358b41b6ee8db9f"} Jun 10 12:55:33.092511 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:33.092461 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.074961236 podStartE2EDuration="2.092445585s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.484349ms" isStatefulPod=false podCreationTimestamp="2026-06-10 12:55:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:55:33.091419936 +0000 UTC m=+394.831634415" watchObservedRunningTime="2026-06-10 12:55:33.092445585 +0000 UTC m=+394.832660065" Jun 10 12:55:33.890474 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:33.890429 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:55:33.996610 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:33.996577 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:55:36.415728 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:36.415697 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:55:44.519616 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:55:44.519574 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" podUID="49614bc3-1721-4b77-90b6-8fcfa6db5ae1" Jun 10 12:55:45.097742 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:45.097706 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:55:45.265889 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:45.265856 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:55:45.367418 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:45.367381 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:55:45.439102 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:45.439058 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:55:46.778614 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:46.778576 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-4lhwz\" (UID: \"49614bc3-1721-4b77-90b6-8fcfa6db5ae1\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:55:46.781220 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:46.781196 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49614bc3-1721-4b77-90b6-8fcfa6db5ae1-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-4lhwz\" (UID: \"49614bc3-1721-4b77-90b6-8fcfa6db5ae1\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:55:46.901489 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:46.901459 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8jmj2\"" Jun 10 12:55:46.908350 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:46.908329 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" Jun 10 12:55:47.061958 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:47.061885 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz"] Jun 10 12:55:47.065524 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:55:47.065496 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49614bc3_1721_4b77_90b6_8fcfa6db5ae1.slice/crio-1d7c5a7c3b2c36e6f71b37f839ec01342eb63248432cb323db09901e57238c6d WatchSource:0}: Error finding container 1d7c5a7c3b2c36e6f71b37f839ec01342eb63248432cb323db09901e57238c6d: Status 404 returned error can't find the container with id 1d7c5a7c3b2c36e6f71b37f839ec01342eb63248432cb323db09901e57238c6d Jun 10 12:55:47.106459 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:47.106425 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" event={"ID":"49614bc3-1721-4b77-90b6-8fcfa6db5ae1","Type":"ContainerStarted","Data":"1d7c5a7c3b2c36e6f71b37f839ec01342eb63248432cb323db09901e57238c6d"} Jun 10 12:55:49.133695 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:49.133655 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" event={"ID":"49614bc3-1721-4b77-90b6-8fcfa6db5ae1","Type":"ContainerStarted","Data":"eb7e69b4a0ceed0a6d7903f333cf0b57f34ea6b8aa47d0fe70037680d2c055d6"} Jun 10 12:55:49.153947 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:49.153899 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-4lhwz" podStartSLOduration=407.005006344 podStartE2EDuration="6m48.153884414s" totalImagesPullingTime="1.14887807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:49:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:55:49.152407669 +0000 UTC m=+410.892622152" watchObservedRunningTime="2026-06-10 12:55:49.153884414 +0000 UTC m=+410.894098895" Jun 10 12:55:49.279849 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:49.279807 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:55:49.354405 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:55:49.354366 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:56:08.941191 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:56:08.941135 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:56:09.093800 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:56:09.093763 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:56:16.515711 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:56:16.515615 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:56:16.579916 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:56:16.579880 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:56:31.415655 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:56:31.415614 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:56:31.431921 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:56:31.431895 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:56:32.320136 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:56:32.320106 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 10 12:56:39.993402 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:56:39.993374 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:56:40.058051 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:56:40.058014 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:56:42.795366 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:56:42.795330 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:56:44.210042 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:56:44.210013 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:56:51.898505 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:56:51.898469 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:56:51.960637 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:56:51.960581 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:56:55.274320 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:56:55.274276 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:56:55.372436 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:56:55.372398 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:56:55.626211 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:56:55.626157 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:56:56.947595 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:56:56.947559 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:56:57.013606 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:56:57.013570 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:57:07.512537 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:57:07.512497 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:57:07.720057 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:57:07.720016 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:57:07.793500 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:57:07.793411 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:57:14.755667 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:57:14.755633 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:57:14.868681 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:57:14.868641 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:57:46.170486 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:57:46.170394 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:57:47.635937 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:57:47.635899 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:57:49.802532 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:57:49.802483 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:57:49.910372 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:57:49.910333 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:57:51.028498 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:57:51.028447 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:57:51.125587 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:57:51.125550 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:57:51.250644 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:57:51.250569 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:57:51.344657 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:57:51.344562 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:57:54.367308 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:57:54.367280 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:57:54.985741 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:57:54.985705 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:57:56.592948 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:57:56.592889 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:57:56.661884 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:57:56.661847 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:00.505481 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:00.505441 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:00.607438 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:00.607404 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:02.960699 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:02.960656 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:04.057639 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:04.057593 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:04.159008 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:04.158973 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:10.378887 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:10.378850 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:10.470058 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:10.470011 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:12.581933 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:12.581880 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:28.049700 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:28.049666 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:28.140769 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:28.140724 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:28.627212 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:28.627136 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:28.980858 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:28.980767 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:29.095567 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.095531 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:29.283397 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.283316 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr"] Jun 10 12:58:29.286673 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.286653 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" Jun 10 12:58:29.291074 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.291055 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 10 12:58:29.291193 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.291113 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qv28b\"" Jun 10 12:58:29.291414 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.291400 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 10 12:58:29.291624 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.291608 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 10 12:58:29.291680 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.291669 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 10 12:58:29.319888 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.319862 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr"] Jun 10 12:58:29.402740 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.402712 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnpjl\" (UniqueName: \"kubernetes.io/projected/ba89288e-b39e-46ba-afc0-e5960356305a-kube-api-access-xnpjl\") pod \"opendatahub-operator-controller-manager-6bb4db9f45-zttsr\" (UID: \"ba89288e-b39e-46ba-afc0-e5960356305a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" Jun 10 12:58:29.402885 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.402749 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba89288e-b39e-46ba-afc0-e5960356305a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bb4db9f45-zttsr\" (UID: \"ba89288e-b39e-46ba-afc0-e5960356305a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" Jun 10 12:58:29.402885 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.402773 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba89288e-b39e-46ba-afc0-e5960356305a-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bb4db9f45-zttsr\" (UID: \"ba89288e-b39e-46ba-afc0-e5960356305a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" Jun 10 12:58:29.402885 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.402795 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba89288e-b39e-46ba-afc0-e5960356305a-manifests\") pod \"opendatahub-operator-controller-manager-6bb4db9f45-zttsr\" (UID: \"ba89288e-b39e-46ba-afc0-e5960356305a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" Jun 10 12:58:29.402885 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.402877 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba89288e-b39e-46ba-afc0-e5960356305a-tmp\") pod \"opendatahub-operator-controller-manager-6bb4db9f45-zttsr\" (UID: \"ba89288e-b39e-46ba-afc0-e5960356305a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" Jun 10 12:58:29.485088 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.485050 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:29.504227 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.504197 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba89288e-b39e-46ba-afc0-e5960356305a-tmp\") pod \"opendatahub-operator-controller-manager-6bb4db9f45-zttsr\" (UID: \"ba89288e-b39e-46ba-afc0-e5960356305a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" Jun 10 12:58:29.504362 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.504244 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnpjl\" (UniqueName: \"kubernetes.io/projected/ba89288e-b39e-46ba-afc0-e5960356305a-kube-api-access-xnpjl\") pod \"opendatahub-operator-controller-manager-6bb4db9f45-zttsr\" (UID: \"ba89288e-b39e-46ba-afc0-e5960356305a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" Jun 10 12:58:29.504362 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.504288 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba89288e-b39e-46ba-afc0-e5960356305a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bb4db9f45-zttsr\" (UID: \"ba89288e-b39e-46ba-afc0-e5960356305a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" Jun 10 12:58:29.504362 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.504321 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba89288e-b39e-46ba-afc0-e5960356305a-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bb4db9f45-zttsr\" (UID: \"ba89288e-b39e-46ba-afc0-e5960356305a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" Jun 10 12:58:29.504362 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.504355 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba89288e-b39e-46ba-afc0-e5960356305a-manifests\") pod \"opendatahub-operator-controller-manager-6bb4db9f45-zttsr\" (UID: \"ba89288e-b39e-46ba-afc0-e5960356305a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" Jun 10 12:58:29.504739 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.504708 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ba89288e-b39e-46ba-afc0-e5960356305a-manifests\") pod \"opendatahub-operator-controller-manager-6bb4db9f45-zttsr\" (UID: \"ba89288e-b39e-46ba-afc0-e5960356305a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" Jun 10 12:58:29.506541 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.506512 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba89288e-b39e-46ba-afc0-e5960356305a-tmp\") pod \"opendatahub-operator-controller-manager-6bb4db9f45-zttsr\" (UID: \"ba89288e-b39e-46ba-afc0-e5960356305a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" Jun 10 12:58:29.506819 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.506799 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ba89288e-b39e-46ba-afc0-e5960356305a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bb4db9f45-zttsr\" (UID: \"ba89288e-b39e-46ba-afc0-e5960356305a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" Jun 10 12:58:29.506908 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.506889 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba89288e-b39e-46ba-afc0-e5960356305a-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bb4db9f45-zttsr\" (UID: \"ba89288e-b39e-46ba-afc0-e5960356305a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" Jun 10 12:58:29.515160 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.515140 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnpjl\" (UniqueName: \"kubernetes.io/projected/ba89288e-b39e-46ba-afc0-e5960356305a-kube-api-access-xnpjl\") pod \"opendatahub-operator-controller-manager-6bb4db9f45-zttsr\" (UID: \"ba89288e-b39e-46ba-afc0-e5960356305a\") " pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" Jun 10 12:58:29.583670 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.583597 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:29.596512 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.596490 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" Jun 10 12:58:29.680793 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.680764 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:29.731010 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.730975 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr"] Jun 10 12:58:29.733533 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:58:29.733511 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba89288e_b39e_46ba_afc0_e5960356305a.slice/crio-9b049cc5341f9911ebc432b3fa7564383e6d565e19b63ed0cacf9201223c512c WatchSource:0}: Error finding container 9b049cc5341f9911ebc432b3fa7564383e6d565e19b63ed0cacf9201223c512c: Status 404 returned error can't find the container with id 9b049cc5341f9911ebc432b3fa7564383e6d565e19b63ed0cacf9201223c512c Jun 10 12:58:29.760649 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.760577 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" event={"ID":"ba89288e-b39e-46ba-afc0-e5960356305a","Type":"ContainerStarted","Data":"9b049cc5341f9911ebc432b3fa7564383e6d565e19b63ed0cacf9201223c512c"} Jun 10 12:58:29.770790 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.770751 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:29.851568 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.851496 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:29.947773 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:29.947742 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:30.043838 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:30.043810 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:32.776466 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:32.776383 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" event={"ID":"ba89288e-b39e-46ba-afc0-e5960356305a","Type":"ContainerStarted","Data":"6c68f9879fdc1fce51f714183c9cd65335fbaa71bdc60c4d65112bc2e4a4a208"} Jun 10 12:58:32.912445 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:32.912401 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:32.990507 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:32.990456 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:33.781705 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:33.781666 2581 generic.go:356] "Generic (PLEG): container finished" podID="ba89288e-b39e-46ba-afc0-e5960356305a" containerID="6c68f9879fdc1fce51f714183c9cd65335fbaa71bdc60c4d65112bc2e4a4a208" exitCode=0 Jun 10 12:58:33.782077 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:33.781745 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" event={"ID":"ba89288e-b39e-46ba-afc0-e5960356305a","Type":"ContainerDied","Data":"6c68f9879fdc1fce51f714183c9cd65335fbaa71bdc60c4d65112bc2e4a4a208"} Jun 10 12:58:34.788261 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:34.788229 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" event={"ID":"ba89288e-b39e-46ba-afc0-e5960356305a","Type":"ContainerStarted","Data":"2e8ea1b058799b9be180f29bf29f60d9d92515572a661efb398d654585d477d6"} Jun 10 12:58:34.788622 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:34.788301 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" Jun 10 12:58:34.827853 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:34.827810 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" podStartSLOduration=1.765633199 podStartE2EDuration="5.827794327s" totalImagesPullingTime="3.088214303s" totalInitContainerRuntime="973.946825ms" isStatefulPod=false podCreationTimestamp="2026-06-10 12:58:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:58:34.827428204 +0000 UTC m=+576.567642685" watchObservedRunningTime="2026-06-10 12:58:34.827794327 +0000 UTC m=+576.568008805" Jun 10 12:58:45.794414 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:45.794383 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6bb4db9f45-zttsr" Jun 10 12:58:47.652664 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:47.652627 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:48.630989 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:48.630957 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-hbpq7"] Jun 10 12:58:48.634037 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:48.634022 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-hbpq7" Jun 10 12:58:48.636939 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:48.636917 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-lq9kt\"" Jun 10 12:58:48.637283 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:48.637269 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 10 12:58:48.644657 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:48.644640 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-hbpq7"] Jun 10 12:58:48.683514 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:48.683475 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4plqk\" (UniqueName: \"kubernetes.io/projected/6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a-kube-api-access-4plqk\") pod \"odh-model-controller-6dc94f689f-hbpq7\" (UID: \"6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a\") " pod="opendatahub/odh-model-controller-6dc94f689f-hbpq7" Jun 10 12:58:48.683863 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:48.683531 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a-cert\") pod \"odh-model-controller-6dc94f689f-hbpq7\" (UID: \"6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a\") " pod="opendatahub/odh-model-controller-6dc94f689f-hbpq7" Jun 10 12:58:48.784566 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:48.784530 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a-cert\") pod \"odh-model-controller-6dc94f689f-hbpq7\" (UID: \"6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a\") " pod="opendatahub/odh-model-controller-6dc94f689f-hbpq7" Jun 10 12:58:48.784745 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:48.784674 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4plqk\" (UniqueName: \"kubernetes.io/projected/6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a-kube-api-access-4plqk\") pod \"odh-model-controller-6dc94f689f-hbpq7\" (UID: \"6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a\") " pod="opendatahub/odh-model-controller-6dc94f689f-hbpq7" Jun 10 12:58:48.784745 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:58:48.784678 2581 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 10 12:58:48.784934 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:58:48.784760 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a-cert podName:6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a nodeName:}" failed. No retries permitted until 2026-06-10 12:58:49.28473794 +0000 UTC m=+591.024952400 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a-cert") pod "odh-model-controller-6dc94f689f-hbpq7" (UID: "6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a") : secret "odh-model-controller-webhook-cert" not found Jun 10 12:58:48.800436 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:48.800415 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4plqk\" (UniqueName: \"kubernetes.io/projected/6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a-kube-api-access-4plqk\") pod \"odh-model-controller-6dc94f689f-hbpq7\" (UID: \"6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a\") " pod="opendatahub/odh-model-controller-6dc94f689f-hbpq7" Jun 10 12:58:48.872032 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:48.871997 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:49.011104 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:49.010992 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:49.187441 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:49.187412 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:49.289853 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:49.289755 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a-cert\") pod \"odh-model-controller-6dc94f689f-hbpq7\" (UID: \"6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a\") " pod="opendatahub/odh-model-controller-6dc94f689f-hbpq7" Jun 10 12:58:49.292431 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:49.292408 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a-cert\") pod \"odh-model-controller-6dc94f689f-hbpq7\" (UID: \"6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a\") " pod="opendatahub/odh-model-controller-6dc94f689f-hbpq7" Jun 10 12:58:49.526219 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:49.526178 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:49.545071 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:49.544988 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-hbpq7" Jun 10 12:58:49.692365 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:49.692336 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-hbpq7"] Jun 10 12:58:49.696503 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:58:49.696468 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c5e63c4_4fec_4cee_8b53_6be5cecb9d4a.slice/crio-5c974f783f6b7e7ba546b0e5d404d519abbdc25202f335b0490debd6b28c6f33 WatchSource:0}: Error finding container 5c974f783f6b7e7ba546b0e5d404d519abbdc25202f335b0490debd6b28c6f33: Status 404 returned error can't find the container with id 5c974f783f6b7e7ba546b0e5d404d519abbdc25202f335b0490debd6b28c6f33 Jun 10 12:58:49.853463 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:49.853377 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-hbpq7" event={"ID":"6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a","Type":"ContainerStarted","Data":"5c974f783f6b7e7ba546b0e5d404d519abbdc25202f335b0490debd6b28c6f33"} Jun 10 12:58:49.870150 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:49.870124 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:52.871211 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:52.871152 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-hbpq7" event={"ID":"6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a","Type":"ContainerStarted","Data":"84a9565b266b7bcbad922ebe376019556300c3da00ad46833e7371338e623707"} Jun 10 12:58:52.871607 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:52.871290 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-6dc94f689f-hbpq7" Jun 10 12:58:52.893676 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:52.893628 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-6dc94f689f-hbpq7" podStartSLOduration=2.218646094 podStartE2EDuration="4.893613412s" totalImagesPullingTime="2.674967318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:58:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:58:52.892443038 +0000 UTC m=+594.632657521" watchObservedRunningTime="2026-06-10 12:58:52.893613412 +0000 UTC m=+594.633827893" Jun 10 12:58:55.814397 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:55.814359 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg"] Jun 10 12:58:55.817690 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:55.817669 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg" Jun 10 12:58:55.820374 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:55.820355 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 10 12:58:55.821687 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:55.821670 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 10 12:58:55.821794 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:55.821750 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 10 12:58:55.821794 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:55.821753 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-9xrhq\"" Jun 10 12:58:55.821904 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:55.821838 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 10 12:58:55.822007 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:55.821994 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 10 12:58:55.829906 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:55.829884 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg"] Jun 10 12:58:55.951528 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:55.951496 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tpdf\" (UniqueName: \"kubernetes.io/projected/5b84ae64-e59e-4712-8f23-1ec53d548e21-kube-api-access-8tpdf\") pod \"lws-controller-manager-7b975d4df8-6fjzg\" (UID: \"5b84ae64-e59e-4712-8f23-1ec53d548e21\") " pod="openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg" Jun 10 12:58:55.951709 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:55.951546 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5b84ae64-e59e-4712-8f23-1ec53d548e21-manager-config\") pod \"lws-controller-manager-7b975d4df8-6fjzg\" (UID: \"5b84ae64-e59e-4712-8f23-1ec53d548e21\") " pod="openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg" Jun 10 12:58:55.951709 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:55.951567 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b84ae64-e59e-4712-8f23-1ec53d548e21-cert\") pod \"lws-controller-manager-7b975d4df8-6fjzg\" (UID: \"5b84ae64-e59e-4712-8f23-1ec53d548e21\") " pod="openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg" Jun 10 12:58:55.951709 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:55.951679 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b84ae64-e59e-4712-8f23-1ec53d548e21-metrics-cert\") pod \"lws-controller-manager-7b975d4df8-6fjzg\" (UID: \"5b84ae64-e59e-4712-8f23-1ec53d548e21\") " pod="openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg" Jun 10 12:58:56.052087 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:56.052056 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5b84ae64-e59e-4712-8f23-1ec53d548e21-manager-config\") pod \"lws-controller-manager-7b975d4df8-6fjzg\" (UID: \"5b84ae64-e59e-4712-8f23-1ec53d548e21\") " pod="openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg" Jun 10 12:58:56.052087 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:56.052093 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b84ae64-e59e-4712-8f23-1ec53d548e21-cert\") pod \"lws-controller-manager-7b975d4df8-6fjzg\" (UID: \"5b84ae64-e59e-4712-8f23-1ec53d548e21\") " pod="openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg" Jun 10 12:58:56.052349 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:56.052124 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b84ae64-e59e-4712-8f23-1ec53d548e21-metrics-cert\") pod \"lws-controller-manager-7b975d4df8-6fjzg\" (UID: \"5b84ae64-e59e-4712-8f23-1ec53d548e21\") " pod="openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg" Jun 10 12:58:56.052349 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:56.052220 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tpdf\" (UniqueName: \"kubernetes.io/projected/5b84ae64-e59e-4712-8f23-1ec53d548e21-kube-api-access-8tpdf\") pod \"lws-controller-manager-7b975d4df8-6fjzg\" (UID: \"5b84ae64-e59e-4712-8f23-1ec53d548e21\") " pod="openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg" Jun 10 12:58:56.052823 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:56.052801 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5b84ae64-e59e-4712-8f23-1ec53d548e21-manager-config\") pod \"lws-controller-manager-7b975d4df8-6fjzg\" (UID: \"5b84ae64-e59e-4712-8f23-1ec53d548e21\") " pod="openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg" Jun 10 12:58:56.054757 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:56.054732 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b84ae64-e59e-4712-8f23-1ec53d548e21-metrics-cert\") pod \"lws-controller-manager-7b975d4df8-6fjzg\" (UID: \"5b84ae64-e59e-4712-8f23-1ec53d548e21\") " pod="openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg" Jun 10 12:58:56.054922 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:56.054906 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b84ae64-e59e-4712-8f23-1ec53d548e21-cert\") pod \"lws-controller-manager-7b975d4df8-6fjzg\" (UID: \"5b84ae64-e59e-4712-8f23-1ec53d548e21\") " pod="openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg" Jun 10 12:58:56.070923 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:56.070868 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tpdf\" (UniqueName: \"kubernetes.io/projected/5b84ae64-e59e-4712-8f23-1ec53d548e21-kube-api-access-8tpdf\") pod \"lws-controller-manager-7b975d4df8-6fjzg\" (UID: \"5b84ae64-e59e-4712-8f23-1ec53d548e21\") " pod="openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg" Jun 10 12:58:56.127119 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:56.127093 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg" Jun 10 12:58:56.267071 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:56.267039 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg"] Jun 10 12:58:56.270609 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:58:56.270576 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b84ae64_e59e_4712_8f23_1ec53d548e21.slice/crio-fbc0732f222f0cf44b3cdbb0c4d321c24541904d50c7513febab991fc66af9a8 WatchSource:0}: Error finding container fbc0732f222f0cf44b3cdbb0c4d321c24541904d50c7513febab991fc66af9a8: Status 404 returned error can't find the container with id fbc0732f222f0cf44b3cdbb0c4d321c24541904d50c7513febab991fc66af9a8 Jun 10 12:58:56.889400 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:56.889370 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg" event={"ID":"5b84ae64-e59e-4712-8f23-1ec53d548e21","Type":"ContainerStarted","Data":"fbc0732f222f0cf44b3cdbb0c4d321c24541904d50c7513febab991fc66af9a8"} Jun 10 12:58:57.479062 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:57.479029 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:57.567854 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:57.567821 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:58.704228 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:58.704200 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgg5q_8a91de05-d0b9-4192-9583-8de8cf91c175/ovn-acl-logging/0.log" Jun 10 12:58:58.704951 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:58.704926 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgg5q_8a91de05-d0b9-4192-9583-8de8cf91c175/ovn-acl-logging/0.log" Jun 10 12:58:59.764999 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:59.764910 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:59.868853 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:59.868818 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:58:59.955878 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:58:59.955837 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:00.100860 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:00.100765 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:00.437010 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:00.436969 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:00.912814 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:00.912780 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg" event={"ID":"5b84ae64-e59e-4712-8f23-1ec53d548e21","Type":"ContainerStarted","Data":"d17656a564c85028e3d21c2d57cb99098ea0e58cf26724cccb452a413a337c0f"} Jun 10 12:59:00.913204 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:00.912922 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg" Jun 10 12:59:01.093793 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:01.093762 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:01.205412 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:01.205321 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:01.297583 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:01.297548 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:02.925811 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:02.925774 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:03.029064 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:03.029025 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:03.614602 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:03.614567 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:03.757477 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:03.757436 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:03.877966 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:03.877886 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-6dc94f689f-hbpq7" Jun 10 12:59:03.927052 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:03.927001 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg" podStartSLOduration=4.679341266 podStartE2EDuration="8.926983523s" totalImagesPullingTime="4.247642257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:58:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:59:00.949074256 +0000 UTC m=+602.689288737" watchObservedRunningTime="2026-06-10 12:59:03.926983523 +0000 UTC m=+605.667198003" Jun 10 12:59:04.614472 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:04.614435 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:06.046830 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:06.046794 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:06.047858 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:06.047836 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-dd7b5d84c-fkx7x"] Jun 10 12:59:06.051406 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:06.051390 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-dd7b5d84c-fkx7x" Jun 10 12:59:06.056715 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:06.056692 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-m87gb\"" Jun 10 12:59:06.076344 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:06.076317 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 10 12:59:06.085130 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:06.085111 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-dd7b5d84c-fkx7x"] Jun 10 12:59:06.152370 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:06.152337 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/97f8a76c-d790-4eae-a95d-c4c8e370ad0c-redirect-config\") pod \"dashboard-redirect-dd7b5d84c-fkx7x\" (UID: \"97f8a76c-d790-4eae-a95d-c4c8e370ad0c\") " pod="opendatahub/dashboard-redirect-dd7b5d84c-fkx7x" Jun 10 12:59:06.152529 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:06.152390 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km67x\" (UniqueName: \"kubernetes.io/projected/97f8a76c-d790-4eae-a95d-c4c8e370ad0c-kube-api-access-km67x\") pod \"dashboard-redirect-dd7b5d84c-fkx7x\" (UID: \"97f8a76c-d790-4eae-a95d-c4c8e370ad0c\") " pod="opendatahub/dashboard-redirect-dd7b5d84c-fkx7x" Jun 10 12:59:06.253764 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:06.253721 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/97f8a76c-d790-4eae-a95d-c4c8e370ad0c-redirect-config\") pod \"dashboard-redirect-dd7b5d84c-fkx7x\" (UID: \"97f8a76c-d790-4eae-a95d-c4c8e370ad0c\") " pod="opendatahub/dashboard-redirect-dd7b5d84c-fkx7x" Jun 10 12:59:06.253764 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:06.253771 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km67x\" (UniqueName: \"kubernetes.io/projected/97f8a76c-d790-4eae-a95d-c4c8e370ad0c-kube-api-access-km67x\") pod \"dashboard-redirect-dd7b5d84c-fkx7x\" (UID: \"97f8a76c-d790-4eae-a95d-c4c8e370ad0c\") " pod="opendatahub/dashboard-redirect-dd7b5d84c-fkx7x" Jun 10 12:59:06.254481 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:06.254462 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/97f8a76c-d790-4eae-a95d-c4c8e370ad0c-redirect-config\") pod \"dashboard-redirect-dd7b5d84c-fkx7x\" (UID: \"97f8a76c-d790-4eae-a95d-c4c8e370ad0c\") " pod="opendatahub/dashboard-redirect-dd7b5d84c-fkx7x" Jun 10 12:59:06.269923 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:06.269896 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-km67x\" (UniqueName: \"kubernetes.io/projected/97f8a76c-d790-4eae-a95d-c4c8e370ad0c-kube-api-access-km67x\") pod \"dashboard-redirect-dd7b5d84c-fkx7x\" (UID: \"97f8a76c-d790-4eae-a95d-c4c8e370ad0c\") " pod="opendatahub/dashboard-redirect-dd7b5d84c-fkx7x" Jun 10 12:59:06.362465 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:06.362438 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-dd7b5d84c-fkx7x" Jun 10 12:59:06.529149 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:06.529119 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-dd7b5d84c-fkx7x"] Jun 10 12:59:06.530611 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:59:06.530580 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97f8a76c_d790_4eae_a95d_c4c8e370ad0c.slice/crio-ebee596b3385134207c92ccec256dc246c3e4d70774fe2f82f7b153e5fd8e43c WatchSource:0}: Error finding container ebee596b3385134207c92ccec256dc246c3e4d70774fe2f82f7b153e5fd8e43c: Status 404 returned error can't find the container with id ebee596b3385134207c92ccec256dc246c3e4d70774fe2f82f7b153e5fd8e43c Jun 10 12:59:06.532119 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:06.532098 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 12:59:06.941264 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:06.941229 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-dd7b5d84c-fkx7x" event={"ID":"97f8a76c-d790-4eae-a95d-c4c8e370ad0c","Type":"ContainerStarted","Data":"ebee596b3385134207c92ccec256dc246c3e4d70774fe2f82f7b153e5fd8e43c"} Jun 10 12:59:11.919291 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:11.919264 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7b975d4df8-6fjzg" Jun 10 12:59:11.968328 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:11.968293 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-dd7b5d84c-fkx7x" event={"ID":"97f8a76c-d790-4eae-a95d-c4c8e370ad0c","Type":"ContainerStarted","Data":"f4c385ed10b8e6454b1696a00ebce5b7634af41784d45c90ec2f2c380ccfb22d"} Jun 10 12:59:14.171717 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:14.171683 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:14.907702 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:14.907661 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:14.973846 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:14.973804 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:25.288996 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:25.288910 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:36.099228 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:36.099188 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:36.197357 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:36.197321 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:36.702730 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:36.702697 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:40.979947 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:40.979893 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:41.081653 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.081606 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:41.252629 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.252519 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-dd7b5d84c-fkx7x" podStartSLOduration=30.84261058 podStartE2EDuration="35.252500753s" totalImagesPullingTime="4.409890173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:59:12.003758045 +0000 UTC m=+613.743972527" watchObservedRunningTime="2026-06-10 12:59:41.252500753 +0000 UTC m=+642.992715276" Jun 10 12:59:41.253383 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.253365 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m"] Jun 10 12:59:41.263585 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.263555 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.266426 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.266399 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-w2jlp\"" Jun 10 12:59:41.266565 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.266435 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 10 12:59:41.273233 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.273209 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m"] Jun 10 12:59:41.289842 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.289801 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:41.343555 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.343524 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.343764 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.343566 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.343764 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.343593 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.343764 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.343613 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.343764 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.343691 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gphp\" (UniqueName: \"kubernetes.io/projected/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-kube-api-access-5gphp\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.343764 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.343748 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.343951 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.343787 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.343951 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.343827 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.343951 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.343842 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.383544 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.383508 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:41.444717 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.444686 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.444717 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.444716 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.444942 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.444738 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.444942 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.444771 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.444942 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.444793 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.444942 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.444811 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.444942 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.444841 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gphp\" (UniqueName: \"kubernetes.io/projected/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-kube-api-access-5gphp\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.444942 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.444896 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.444942 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.444930 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.445347 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.445225 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.445407 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.445351 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.445566 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.445434 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.445633 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.445566 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.445633 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.445521 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.447016 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.446997 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.447294 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.447275 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.457021 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.456993 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.457264 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.457242 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gphp\" (UniqueName: \"kubernetes.io/projected/2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d-kube-api-access-5gphp\") pod \"data-science-gateway-data-science-gateway-class-5b556588bck6g5m\" (UID: \"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.485881 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.485838 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:41.575912 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.575821 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:41.715260 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:41.715237 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m"] Jun 10 12:59:41.717802 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:59:41.717773 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fee2f7e_d91a_4f9b_8848_7f9c347bcf0d.slice/crio-4e77370d4121449cf2404509e60919b44c0b26fd0a670c83622d798c700152b9 WatchSource:0}: Error finding container 4e77370d4121449cf2404509e60919b44c0b26fd0a670c83622d798c700152b9: Status 404 returned error can't find the container with id 4e77370d4121449cf2404509e60919b44c0b26fd0a670c83622d798c700152b9 Jun 10 12:59:42.098387 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:42.098350 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" event={"ID":"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d","Type":"ContainerStarted","Data":"4e77370d4121449cf2404509e60919b44c0b26fd0a670c83622d798c700152b9"} Jun 10 12:59:44.660203 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:44.660144 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jun 10 12:59:44.660472 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:44.660248 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jun 10 12:59:44.660472 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:44.660280 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jun 10 12:59:45.115612 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:45.115579 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" event={"ID":"2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d","Type":"ContainerStarted","Data":"a1f4dd9577125b02890338458f9c5e9beb3a50483d5717716b702b2d7259efbc"} Jun 10 12:59:45.140384 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:45.140337 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" podStartSLOduration=1.200190548 podStartE2EDuration="4.140323103s" totalImagesPullingTime="2.940132555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:59:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:59:45.139019815 +0000 UTC m=+646.879234297" watchObservedRunningTime="2026-06-10 12:59:45.140323103 +0000 UTC m=+646.880537584" Jun 10 12:59:45.424477 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:45.424387 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:45.576381 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:45.576344 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:45.581040 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:45.581015 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:46.120301 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:46.120269 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:46.121327 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:46.121309 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bck6g5m" Jun 10 12:59:49.267772 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:49.267737 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:49.360310 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:49.360277 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:49.382036 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:49.382011 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d9t7q"] Jun 10 12:59:49.385379 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:49.385364 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-d9t7q" Jun 10 12:59:49.388431 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:49.388402 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 10 12:59:49.389282 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:49.389258 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-sjcld\"" Jun 10 12:59:49.389522 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:49.389342 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 10 12:59:49.395226 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:49.395201 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d9t7q"] Jun 10 12:59:49.433665 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:49.433633 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:49.517028 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:49.516993 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh9nm\" (UniqueName: \"kubernetes.io/projected/08e23455-1392-4edd-978e-d0e71b288928-kube-api-access-nh9nm\") pod \"kuadrant-operator-catalog-d9t7q\" (UID: \"08e23455-1392-4edd-978e-d0e71b288928\") " pod="kuadrant-system/kuadrant-operator-catalog-d9t7q" Jun 10 12:59:49.618438 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:49.618408 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh9nm\" (UniqueName: \"kubernetes.io/projected/08e23455-1392-4edd-978e-d0e71b288928-kube-api-access-nh9nm\") pod \"kuadrant-operator-catalog-d9t7q\" (UID: \"08e23455-1392-4edd-978e-d0e71b288928\") " pod="kuadrant-system/kuadrant-operator-catalog-d9t7q" Jun 10 12:59:49.629723 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:49.629698 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh9nm\" (UniqueName: \"kubernetes.io/projected/08e23455-1392-4edd-978e-d0e71b288928-kube-api-access-nh9nm\") pod \"kuadrant-operator-catalog-d9t7q\" (UID: \"08e23455-1392-4edd-978e-d0e71b288928\") " pod="kuadrant-system/kuadrant-operator-catalog-d9t7q" Jun 10 12:59:49.696825 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:49.696782 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-d9t7q" Jun 10 12:59:49.754777 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:49.754750 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d9t7q"] Jun 10 12:59:49.831556 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:49.831401 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d9t7q"] Jun 10 12:59:49.833924 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:59:49.833899 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08e23455_1392_4edd_978e_d0e71b288928.slice/crio-dc5976bc17e38f52156cec9821e16c933e4f61e2d84f40f9ac527055dbcf34e3 WatchSource:0}: Error finding container dc5976bc17e38f52156cec9821e16c933e4f61e2d84f40f9ac527055dbcf34e3: Status 404 returned error can't find the container with id dc5976bc17e38f52156cec9821e16c933e4f61e2d84f40f9ac527055dbcf34e3 Jun 10 12:59:49.953737 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:49.953646 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gwwvm"] Jun 10 12:59:49.958090 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:49.958069 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gwwvm" Jun 10 12:59:49.968260 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:49.968238 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gwwvm"] Jun 10 12:59:50.021710 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:50.021679 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxwfl\" (UniqueName: \"kubernetes.io/projected/553e33c9-29b7-4aff-8887-4f69997c0a84-kube-api-access-hxwfl\") pod \"kuadrant-operator-catalog-gwwvm\" (UID: \"553e33c9-29b7-4aff-8887-4f69997c0a84\") " pod="kuadrant-system/kuadrant-operator-catalog-gwwvm" Jun 10 12:59:50.122260 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:50.122227 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxwfl\" (UniqueName: \"kubernetes.io/projected/553e33c9-29b7-4aff-8887-4f69997c0a84-kube-api-access-hxwfl\") pod \"kuadrant-operator-catalog-gwwvm\" (UID: \"553e33c9-29b7-4aff-8887-4f69997c0a84\") " pod="kuadrant-system/kuadrant-operator-catalog-gwwvm" Jun 10 12:59:50.132738 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:50.132711 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxwfl\" (UniqueName: \"kubernetes.io/projected/553e33c9-29b7-4aff-8887-4f69997c0a84-kube-api-access-hxwfl\") pod \"kuadrant-operator-catalog-gwwvm\" (UID: \"553e33c9-29b7-4aff-8887-4f69997c0a84\") " pod="kuadrant-system/kuadrant-operator-catalog-gwwvm" Jun 10 12:59:50.141616 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:50.141452 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-d9t7q" event={"ID":"08e23455-1392-4edd-978e-d0e71b288928","Type":"ContainerStarted","Data":"dc5976bc17e38f52156cec9821e16c933e4f61e2d84f40f9ac527055dbcf34e3"} Jun 10 12:59:50.182821 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:50.182782 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:50.267859 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:50.267798 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gwwvm" Jun 10 12:59:50.385945 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:50.385908 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 12:59:50.419030 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:50.419000 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gwwvm"] Jun 10 12:59:50.503957 ip-10-0-128-230 kubenswrapper[2581]: W0610 12:59:50.503923 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod553e33c9_29b7_4aff_8887_4f69997c0a84.slice/crio-09c1bfbfd567de88b23d22ded41c8b4f936145e30e385ece65a44f1818456253 WatchSource:0}: Error finding container 09c1bfbfd567de88b23d22ded41c8b4f936145e30e385ece65a44f1818456253: Status 404 returned error can't find the container with id 09c1bfbfd567de88b23d22ded41c8b4f936145e30e385ece65a44f1818456253 Jun 10 12:59:51.149346 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:51.149309 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gwwvm" event={"ID":"553e33c9-29b7-4aff-8887-4f69997c0a84","Type":"ContainerStarted","Data":"09c1bfbfd567de88b23d22ded41c8b4f936145e30e385ece65a44f1818456253"} Jun 10 12:59:52.155445 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:52.155404 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gwwvm" event={"ID":"553e33c9-29b7-4aff-8887-4f69997c0a84","Type":"ContainerStarted","Data":"d9413b86f025e9a9ad7668bf2ad08805c2d43356f9d1db7200e470bd2a631b70"} Jun 10 12:59:52.156836 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:52.156813 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-d9t7q" event={"ID":"08e23455-1392-4edd-978e-d0e71b288928","Type":"ContainerStarted","Data":"b24d969c9bb3dbe8d87a9181ccad59f71b4756d169fa2868cfd6f8c942d41c04"} Jun 10 12:59:52.156959 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:52.156883 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-d9t7q" podUID="08e23455-1392-4edd-978e-d0e71b288928" containerName="registry-server" containerID="cri-o://b24d969c9bb3dbe8d87a9181ccad59f71b4756d169fa2868cfd6f8c942d41c04" gracePeriod=2 Jun 10 12:59:52.175990 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:52.175945 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-gwwvm" podStartSLOduration=1.861113881 podStartE2EDuration="3.175931341s" totalImagesPullingTime="1.31481746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 12:59:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 12:59:52.174400072 +0000 UTC m=+653.914614553" watchObservedRunningTime="2026-06-10 12:59:52.175931341 +0000 UTC m=+653.916145800" Jun 10 12:59:52.399005 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:52.398983 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-d9t7q" Jun 10 12:59:52.547987 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:52.547905 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nh9nm\" (UniqueName: \"kubernetes.io/projected/08e23455-1392-4edd-978e-d0e71b288928-kube-api-access-nh9nm\") pod \"08e23455-1392-4edd-978e-d0e71b288928\" (UID: \"08e23455-1392-4edd-978e-d0e71b288928\") " Jun 10 12:59:52.550230 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:52.550206 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08e23455-1392-4edd-978e-d0e71b288928-kube-api-access-nh9nm" (OuterVolumeSpecName: "kube-api-access-nh9nm") pod "08e23455-1392-4edd-978e-d0e71b288928" (UID: "08e23455-1392-4edd-978e-d0e71b288928"). InnerVolumeSpecName "kube-api-access-nh9nm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 12:59:52.648796 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:52.648766 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nh9nm\" (UniqueName: \"kubernetes.io/projected/08e23455-1392-4edd-978e-d0e71b288928-kube-api-access-nh9nm\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 12:59:53.162668 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:53.162633 2581 generic.go:356] "Generic (PLEG): container finished" podID="08e23455-1392-4edd-978e-d0e71b288928" containerID="b24d969c9bb3dbe8d87a9181ccad59f71b4756d169fa2868cfd6f8c942d41c04" exitCode=0 Jun 10 12:59:53.163095 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:53.162678 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-d9t7q" Jun 10 12:59:53.163095 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:53.162718 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-d9t7q" event={"ID":"08e23455-1392-4edd-978e-d0e71b288928","Type":"ContainerDied","Data":"b24d969c9bb3dbe8d87a9181ccad59f71b4756d169fa2868cfd6f8c942d41c04"} Jun 10 12:59:53.163095 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:53.162769 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-d9t7q" event={"ID":"08e23455-1392-4edd-978e-d0e71b288928","Type":"ContainerDied","Data":"dc5976bc17e38f52156cec9821e16c933e4f61e2d84f40f9ac527055dbcf34e3"} Jun 10 12:59:53.163095 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:53.162789 2581 scope.go:117] "RemoveContainer" containerID="b24d969c9bb3dbe8d87a9181ccad59f71b4756d169fa2868cfd6f8c942d41c04" Jun 10 12:59:53.171632 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:53.171617 2581 scope.go:117] "RemoveContainer" containerID="b24d969c9bb3dbe8d87a9181ccad59f71b4756d169fa2868cfd6f8c942d41c04" Jun 10 12:59:53.171877 ip-10-0-128-230 kubenswrapper[2581]: E0610 12:59:53.171861 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b24d969c9bb3dbe8d87a9181ccad59f71b4756d169fa2868cfd6f8c942d41c04\": container with ID starting with b24d969c9bb3dbe8d87a9181ccad59f71b4756d169fa2868cfd6f8c942d41c04 not found: ID does not exist" containerID="b24d969c9bb3dbe8d87a9181ccad59f71b4756d169fa2868cfd6f8c942d41c04" Jun 10 12:59:53.171934 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:53.171886 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b24d969c9bb3dbe8d87a9181ccad59f71b4756d169fa2868cfd6f8c942d41c04"} err="failed to get container status \"b24d969c9bb3dbe8d87a9181ccad59f71b4756d169fa2868cfd6f8c942d41c04\": rpc error: code = NotFound desc = could not find container \"b24d969c9bb3dbe8d87a9181ccad59f71b4756d169fa2868cfd6f8c942d41c04\": container with ID starting with b24d969c9bb3dbe8d87a9181ccad59f71b4756d169fa2868cfd6f8c942d41c04 not found: ID does not exist" Jun 10 12:59:53.181265 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:53.181244 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d9t7q"] Jun 10 12:59:53.188051 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:53.188030 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-d9t7q"] Jun 10 12:59:54.717530 ip-10-0-128-230 kubenswrapper[2581]: I0610 12:59:54.717501 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08e23455-1392-4edd-978e-d0e71b288928" path="/var/lib/kubelet/pods/08e23455-1392-4edd-978e-d0e71b288928/volumes" Jun 10 13:00:00.267977 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:00.267939 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-gwwvm" Jun 10 13:00:00.268395 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:00.268011 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-gwwvm" Jun 10 13:00:00.290434 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:00.290410 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-gwwvm" Jun 10 13:00:01.223786 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:01.223751 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-gwwvm" Jun 10 13:00:13.283940 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:13.283900 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:15.522641 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:15.522599 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq"] Jun 10 13:00:15.523019 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:15.522949 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08e23455-1392-4edd-978e-d0e71b288928" containerName="registry-server" Jun 10 13:00:15.523019 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:15.522960 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="08e23455-1392-4edd-978e-d0e71b288928" containerName="registry-server" Jun 10 13:00:15.523019 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:15.523016 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="08e23455-1392-4edd-978e-d0e71b288928" containerName="registry-server" Jun 10 13:00:15.532543 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:15.532517 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" Jun 10 13:00:15.538758 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:15.538730 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-w4frs\"" Jun 10 13:00:15.541246 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:15.541221 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq"] Jun 10 13:00:15.648661 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:15.648627 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdrl2\" (UniqueName: \"kubernetes.io/projected/72e8ec98-9362-407e-9030-0a4284c9c1ca-kube-api-access-xdrl2\") pod \"limitador-operator-controller-manager-859d7b78bd-srktq\" (UID: \"72e8ec98-9362-407e-9030-0a4284c9c1ca\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" Jun 10 13:00:15.749509 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:15.749470 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdrl2\" (UniqueName: \"kubernetes.io/projected/72e8ec98-9362-407e-9030-0a4284c9c1ca-kube-api-access-xdrl2\") pod \"limitador-operator-controller-manager-859d7b78bd-srktq\" (UID: \"72e8ec98-9362-407e-9030-0a4284c9c1ca\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" Jun 10 13:00:15.769292 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:15.769258 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdrl2\" (UniqueName: \"kubernetes.io/projected/72e8ec98-9362-407e-9030-0a4284c9c1ca-kube-api-access-xdrl2\") pod \"limitador-operator-controller-manager-859d7b78bd-srktq\" (UID: \"72e8ec98-9362-407e-9030-0a4284c9c1ca\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" Jun 10 13:00:15.843833 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:15.843729 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" Jun 10 13:00:15.978202 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:15.978154 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq"] Jun 10 13:00:15.981069 ip-10-0-128-230 kubenswrapper[2581]: W0610 13:00:15.981039 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72e8ec98_9362_407e_9030_0a4284c9c1ca.slice/crio-e34aa975a29757b787dfb2d8cf13f592abb58c1b0b4b07e4a130c2e4a0ba7ce0 WatchSource:0}: Error finding container e34aa975a29757b787dfb2d8cf13f592abb58c1b0b4b07e4a130c2e4a0ba7ce0: Status 404 returned error can't find the container with id e34aa975a29757b787dfb2d8cf13f592abb58c1b0b4b07e4a130c2e4a0ba7ce0 Jun 10 13:00:16.270419 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:16.270384 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" event={"ID":"72e8ec98-9362-407e-9030-0a4284c9c1ca","Type":"ContainerStarted","Data":"e34aa975a29757b787dfb2d8cf13f592abb58c1b0b4b07e4a130c2e4a0ba7ce0"} Jun 10 13:00:16.679826 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:16.679791 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:16.945994 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:16.945899 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:18.281216 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:18.281180 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" event={"ID":"72e8ec98-9362-407e-9030-0a4284c9c1ca","Type":"ContainerStarted","Data":"32ac4661941ab9d244c5ecbb665f0d9d61b2ea69846b69b5a31443981f37af77"} Jun 10 13:00:18.281644 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:18.281237 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" Jun 10 13:00:18.308676 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:18.308633 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" podStartSLOduration=1.702283684 podStartE2EDuration="3.308618143s" totalImagesPullingTime="1.606334459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:00:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:00:18.30772667 +0000 UTC m=+680.047941152" watchObservedRunningTime="2026-06-10 13:00:18.308618143 +0000 UTC m=+680.048832617" Jun 10 13:00:20.806725 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:20.806689 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp"] Jun 10 13:00:20.810554 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:20.810536 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:20.813086 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:20.813066 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2ndqb\"" Jun 10 13:00:20.822733 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:20.822705 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp"] Jun 10 13:00:20.886313 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:20.886278 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47n6v\" (UniqueName: \"kubernetes.io/projected/290f9654-ba81-4d5f-bf63-9ff58d0a408b-kube-api-access-47n6v\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l9wqp\" (UID: \"290f9654-ba81-4d5f-bf63-9ff58d0a408b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:20.886483 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:20.886335 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/290f9654-ba81-4d5f-bf63-9ff58d0a408b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l9wqp\" (UID: \"290f9654-ba81-4d5f-bf63-9ff58d0a408b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:20.987609 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:20.987571 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/290f9654-ba81-4d5f-bf63-9ff58d0a408b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l9wqp\" (UID: \"290f9654-ba81-4d5f-bf63-9ff58d0a408b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:20.987779 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:20.987695 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47n6v\" (UniqueName: \"kubernetes.io/projected/290f9654-ba81-4d5f-bf63-9ff58d0a408b-kube-api-access-47n6v\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l9wqp\" (UID: \"290f9654-ba81-4d5f-bf63-9ff58d0a408b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:20.988050 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:20.988026 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/290f9654-ba81-4d5f-bf63-9ff58d0a408b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l9wqp\" (UID: \"290f9654-ba81-4d5f-bf63-9ff58d0a408b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:21.001052 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:21.001028 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47n6v\" (UniqueName: \"kubernetes.io/projected/290f9654-ba81-4d5f-bf63-9ff58d0a408b-kube-api-access-47n6v\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l9wqp\" (UID: \"290f9654-ba81-4d5f-bf63-9ff58d0a408b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:21.128231 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:21.128192 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:21.272143 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:21.272117 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp"] Jun 10 13:00:21.275091 ip-10-0-128-230 kubenswrapper[2581]: W0610 13:00:21.275057 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod290f9654_ba81_4d5f_bf63_9ff58d0a408b.slice/crio-457d4fe03f95ed73c7f0ddda44f2d6cd7c3b8e3958934039841df11cb55634ce WatchSource:0}: Error finding container 457d4fe03f95ed73c7f0ddda44f2d6cd7c3b8e3958934039841df11cb55634ce: Status 404 returned error can't find the container with id 457d4fe03f95ed73c7f0ddda44f2d6cd7c3b8e3958934039841df11cb55634ce Jun 10 13:00:21.297036 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:21.297005 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" event={"ID":"290f9654-ba81-4d5f-bf63-9ff58d0a408b","Type":"ContainerStarted","Data":"457d4fe03f95ed73c7f0ddda44f2d6cd7c3b8e3958934039841df11cb55634ce"} Jun 10 13:00:21.931260 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:21.931225 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:22.329654 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:22.329568 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:24.489955 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:24.489913 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:24.730824 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:24.730755 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:24.915828 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:24.915762 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:26.324507 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:26.324475 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" event={"ID":"290f9654-ba81-4d5f-bf63-9ff58d0a408b","Type":"ContainerStarted","Data":"bae76b58875c30ac7b005444a50d8df443fa2afb7a94287fde90ad90566d9ab0"} Jun 10 13:00:26.324896 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:26.324594 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:26.366914 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:26.366856 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" podStartSLOduration=2.270314016 podStartE2EDuration="6.366840794s" totalImagesPullingTime="4.096526778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:00:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:00:26.366725495 +0000 UTC m=+688.106939976" watchObservedRunningTime="2026-06-10 13:00:26.366840794 +0000 UTC m=+688.107055276" Jun 10 13:00:27.132549 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:27.132519 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:27.228687 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:27.228653 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:28.206934 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:28.206891 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:28.646648 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:28.646610 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:29.089160 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:29.089069 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:29.192554 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:29.192513 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:29.288028 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:29.287999 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" Jun 10 13:00:32.483234 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:32.483201 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:32.577020 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:32.576983 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:37.331161 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:37.331133 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:38.668543 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:38.668506 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:38.765143 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:38.765104 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:39.332520 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.332490 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp"] Jun 10 13:00:39.332739 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.332693 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" podUID="290f9654-ba81-4d5f-bf63-9ff58d0a408b" containerName="manager" containerID="cri-o://bae76b58875c30ac7b005444a50d8df443fa2afb7a94287fde90ad90566d9ab0" gracePeriod=2 Jun 10 13:00:39.338602 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.338471 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp"] Jun 10 13:00:39.355703 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.355673 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq"] Jun 10 13:00:39.356073 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.355899 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" podUID="72e8ec98-9362-407e-9030-0a4284c9c1ca" containerName="manager" containerID="cri-o://32ac4661941ab9d244c5ecbb665f0d9d61b2ea69846b69b5a31443981f37af77" gracePeriod=2 Jun 10 13:00:39.366101 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.366073 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr"] Jun 10 13:00:39.366502 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.366485 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="290f9654-ba81-4d5f-bf63-9ff58d0a408b" containerName="manager" Jun 10 13:00:39.366587 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.366503 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="290f9654-ba81-4d5f-bf63-9ff58d0a408b" containerName="manager" Jun 10 13:00:39.366647 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.366596 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="290f9654-ba81-4d5f-bf63-9ff58d0a408b" containerName="manager" Jun 10 13:00:39.369993 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.369974 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr" Jun 10 13:00:39.374214 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.374193 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq"] Jun 10 13:00:39.392409 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:39.392372 2581 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"72e8ec98-9362-407e-9030-0a4284c9c1ca\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-srktq\": pods \"limitador-operator-controller-manager-859d7b78bd-srktq\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" Jun 10 13:00:39.394913 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.394887 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr"] Jun 10 13:00:39.395115 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:39.395090 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l9wqp\" is forbidden: User \"system:node:ip-10-0-128-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-230.ec2.internal' and this object" podUID="290f9654-ba81-4d5f-bf63-9ff58d0a408b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:39.397329 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.397305 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vk784"] Jun 10 13:00:39.397430 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:39.397334 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-srktq\" is forbidden: User \"system:node:ip-10-0-128-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-230.ec2.internal' and this object" podUID="72e8ec98-9362-407e-9030-0a4284c9c1ca" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" Jun 10 13:00:39.397850 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.397828 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72e8ec98-9362-407e-9030-0a4284c9c1ca" containerName="manager" Jun 10 13:00:39.397944 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.397853 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="72e8ec98-9362-407e-9030-0a4284c9c1ca" containerName="manager" Jun 10 13:00:39.397944 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.397941 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="72e8ec98-9362-407e-9030-0a4284c9c1ca" containerName="manager" Jun 10 13:00:39.399553 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:39.399529 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l9wqp\" is forbidden: User \"system:node:ip-10-0-128-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-230.ec2.internal' and this object" podUID="290f9654-ba81-4d5f-bf63-9ff58d0a408b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:39.401540 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.401523 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vk784" Jun 10 13:00:39.413541 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.413522 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vk784"] Jun 10 13:00:39.425816 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:39.425788 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-srktq\" is forbidden: User \"system:node:ip-10-0-128-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-230.ec2.internal' and this object" podUID="72e8ec98-9362-407e-9030-0a4284c9c1ca" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" Jun 10 13:00:39.427989 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:39.427967 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l9wqp\" is forbidden: User \"system:node:ip-10-0-128-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-230.ec2.internal' and this object" podUID="290f9654-ba81-4d5f-bf63-9ff58d0a408b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:39.455864 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.455840 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z6fd\" (UniqueName: \"kubernetes.io/projected/7c46c69f-3d3c-408f-a7b6-ba558717e019-kube-api-access-6z6fd\") pod \"limitador-operator-controller-manager-859d7b78bd-vk784\" (UID: \"7c46c69f-3d3c-408f-a7b6-ba558717e019\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vk784" Jun 10 13:00:39.455973 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.455960 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8c4e12e-abfc-4be7-9b75-41c975e823bd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hd5gr\" (UID: \"e8c4e12e-abfc-4be7-9b75-41c975e823bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr" Jun 10 13:00:39.456019 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.455983 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbrnh\" (UniqueName: \"kubernetes.io/projected/e8c4e12e-abfc-4be7-9b75-41c975e823bd-kube-api-access-hbrnh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hd5gr\" (UID: \"e8c4e12e-abfc-4be7-9b75-41c975e823bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr" Jun 10 13:00:39.556655 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.556619 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8c4e12e-abfc-4be7-9b75-41c975e823bd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hd5gr\" (UID: \"e8c4e12e-abfc-4be7-9b75-41c975e823bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr" Jun 10 13:00:39.556836 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.556671 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbrnh\" (UniqueName: \"kubernetes.io/projected/e8c4e12e-abfc-4be7-9b75-41c975e823bd-kube-api-access-hbrnh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hd5gr\" (UID: \"e8c4e12e-abfc-4be7-9b75-41c975e823bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr" Jun 10 13:00:39.556836 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.556710 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z6fd\" (UniqueName: \"kubernetes.io/projected/7c46c69f-3d3c-408f-a7b6-ba558717e019-kube-api-access-6z6fd\") pod \"limitador-operator-controller-manager-859d7b78bd-vk784\" (UID: \"7c46c69f-3d3c-408f-a7b6-ba558717e019\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vk784" Jun 10 13:00:39.557085 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.557058 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8c4e12e-abfc-4be7-9b75-41c975e823bd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hd5gr\" (UID: \"e8c4e12e-abfc-4be7-9b75-41c975e823bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr" Jun 10 13:00:39.574774 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.574734 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbrnh\" (UniqueName: \"kubernetes.io/projected/e8c4e12e-abfc-4be7-9b75-41c975e823bd-kube-api-access-hbrnh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hd5gr\" (UID: \"e8c4e12e-abfc-4be7-9b75-41c975e823bd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr" Jun 10 13:00:39.575202 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.575146 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z6fd\" (UniqueName: \"kubernetes.io/projected/7c46c69f-3d3c-408f-a7b6-ba558717e019-kube-api-access-6z6fd\") pod \"limitador-operator-controller-manager-859d7b78bd-vk784\" (UID: \"7c46c69f-3d3c-408f-a7b6-ba558717e019\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vk784" Jun 10 13:00:39.602478 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.602458 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:39.604736 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:39.604711 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-srktq\" is forbidden: User \"system:node:ip-10-0-128-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-230.ec2.internal' and this object" podUID="72e8ec98-9362-407e-9030-0a4284c9c1ca" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" Jun 10 13:00:39.604818 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.604720 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" Jun 10 13:00:39.609498 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:39.609474 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l9wqp\" is forbidden: User \"system:node:ip-10-0-128-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-230.ec2.internal' and this object" podUID="290f9654-ba81-4d5f-bf63-9ff58d0a408b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:39.612005 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:39.611984 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-srktq\" is forbidden: User \"system:node:ip-10-0-128-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-230.ec2.internal' and this object" podUID="72e8ec98-9362-407e-9030-0a4284c9c1ca" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" Jun 10 13:00:39.616752 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:39.616728 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l9wqp\" is forbidden: User \"system:node:ip-10-0-128-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-230.ec2.internal' and this object" podUID="290f9654-ba81-4d5f-bf63-9ff58d0a408b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:39.657268 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.657238 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/290f9654-ba81-4d5f-bf63-9ff58d0a408b-extensions-socket-volume\") pod \"290f9654-ba81-4d5f-bf63-9ff58d0a408b\" (UID: \"290f9654-ba81-4d5f-bf63-9ff58d0a408b\") " Jun 10 13:00:39.657398 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.657350 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdrl2\" (UniqueName: \"kubernetes.io/projected/72e8ec98-9362-407e-9030-0a4284c9c1ca-kube-api-access-xdrl2\") pod \"72e8ec98-9362-407e-9030-0a4284c9c1ca\" (UID: \"72e8ec98-9362-407e-9030-0a4284c9c1ca\") " Jun 10 13:00:39.657398 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.657375 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47n6v\" (UniqueName: \"kubernetes.io/projected/290f9654-ba81-4d5f-bf63-9ff58d0a408b-kube-api-access-47n6v\") pod \"290f9654-ba81-4d5f-bf63-9ff58d0a408b\" (UID: \"290f9654-ba81-4d5f-bf63-9ff58d0a408b\") " Jun 10 13:00:39.657774 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.657749 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/290f9654-ba81-4d5f-bf63-9ff58d0a408b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "290f9654-ba81-4d5f-bf63-9ff58d0a408b" (UID: "290f9654-ba81-4d5f-bf63-9ff58d0a408b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 13:00:39.659632 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.659615 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/290f9654-ba81-4d5f-bf63-9ff58d0a408b-kube-api-access-47n6v" (OuterVolumeSpecName: "kube-api-access-47n6v") pod "290f9654-ba81-4d5f-bf63-9ff58d0a408b" (UID: "290f9654-ba81-4d5f-bf63-9ff58d0a408b"). InnerVolumeSpecName "kube-api-access-47n6v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:00:39.659750 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.659663 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72e8ec98-9362-407e-9030-0a4284c9c1ca-kube-api-access-xdrl2" (OuterVolumeSpecName: "kube-api-access-xdrl2") pod "72e8ec98-9362-407e-9030-0a4284c9c1ca" (UID: "72e8ec98-9362-407e-9030-0a4284c9c1ca"). InnerVolumeSpecName "kube-api-access-xdrl2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:00:39.758585 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.758535 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdrl2\" (UniqueName: \"kubernetes.io/projected/72e8ec98-9362-407e-9030-0a4284c9c1ca-kube-api-access-xdrl2\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:00:39.758585 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.758579 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47n6v\" (UniqueName: \"kubernetes.io/projected/290f9654-ba81-4d5f-bf63-9ff58d0a408b-kube-api-access-47n6v\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:00:39.758585 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.758589 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/290f9654-ba81-4d5f-bf63-9ff58d0a408b-extensions-socket-volume\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:00:39.773879 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.773854 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr" Jun 10 13:00:39.782017 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.782001 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vk784" Jun 10 13:00:39.934719 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.934620 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr"] Jun 10 13:00:39.937656 ip-10-0-128-230 kubenswrapper[2581]: W0610 13:00:39.937630 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c4e12e_abfc_4be7_9b75_41c975e823bd.slice/crio-6fab53435ded79c807f6b7b06d019f61bb78ea99f8e7914d865be25f0ef92927 WatchSource:0}: Error finding container 6fab53435ded79c807f6b7b06d019f61bb78ea99f8e7914d865be25f0ef92927: Status 404 returned error can't find the container with id 6fab53435ded79c807f6b7b06d019f61bb78ea99f8e7914d865be25f0ef92927 Jun 10 13:00:39.955137 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:39.955116 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vk784"] Jun 10 13:00:39.958330 ip-10-0-128-230 kubenswrapper[2581]: W0610 13:00:39.958306 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c46c69f_3d3c_408f_a7b6_ba558717e019.slice/crio-2cd6eb808683fa7f0b5ecd4e55f34a5edf71b27b4206e4ae9f8b07c8da968e07 WatchSource:0}: Error finding container 2cd6eb808683fa7f0b5ecd4e55f34a5edf71b27b4206e4ae9f8b07c8da968e07: Status 404 returned error can't find the container with id 2cd6eb808683fa7f0b5ecd4e55f34a5edf71b27b4206e4ae9f8b07c8da968e07 Jun 10 13:00:40.327211 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.327162 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb"] Jun 10 13:00:40.330655 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.330639 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb" Jun 10 13:00:40.333094 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:40.333060 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l9wqp\" is forbidden: User \"system:node:ip-10-0-128-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-230.ec2.internal' and this object" podUID="290f9654-ba81-4d5f-bf63-9ff58d0a408b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:40.350401 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.350370 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb"] Jun 10 13:00:40.386578 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:40.386539 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-srktq\" is forbidden: User \"system:node:ip-10-0-128-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-230.ec2.internal' and this object" podUID="72e8ec98-9362-407e-9030-0a4284c9c1ca" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" Jun 10 13:00:40.391971 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.391948 2581 generic.go:356] "Generic (PLEG): container finished" podID="72e8ec98-9362-407e-9030-0a4284c9c1ca" containerID="32ac4661941ab9d244c5ecbb665f0d9d61b2ea69846b69b5a31443981f37af77" exitCode=0 Jun 10 13:00:40.392098 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.392035 2581 scope.go:117] "RemoveContainer" containerID="32ac4661941ab9d244c5ecbb665f0d9d61b2ea69846b69b5a31443981f37af77" Jun 10 13:00:40.392151 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.392049 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" Jun 10 13:00:40.393895 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.393862 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr" event={"ID":"e8c4e12e-abfc-4be7-9b75-41c975e823bd","Type":"ContainerStarted","Data":"0b8105e61e9e5c7da263bde9f0da030c39a728dfd4c8d8354cf4725a593b682d"} Jun 10 13:00:40.394001 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.393916 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr" event={"ID":"e8c4e12e-abfc-4be7-9b75-41c975e823bd","Type":"ContainerStarted","Data":"6fab53435ded79c807f6b7b06d019f61bb78ea99f8e7914d865be25f0ef92927"} Jun 10 13:00:40.394001 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.393947 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr" Jun 10 13:00:40.395087 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:40.395058 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-srktq\" is forbidden: User \"system:node:ip-10-0-128-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-230.ec2.internal' and this object" podUID="72e8ec98-9362-407e-9030-0a4284c9c1ca" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" Jun 10 13:00:40.395410 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.395387 2581 generic.go:356] "Generic (PLEG): container finished" podID="290f9654-ba81-4d5f-bf63-9ff58d0a408b" containerID="bae76b58875c30ac7b005444a50d8df443fa2afb7a94287fde90ad90566d9ab0" exitCode=0 Jun 10 13:00:40.395506 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.395424 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:40.397206 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.397156 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vk784" event={"ID":"7c46c69f-3d3c-408f-a7b6-ba558717e019","Type":"ContainerStarted","Data":"6eebb8ad5b5c994bbd84fc6c4e6b13f6083dad8b96339c27f914fba09f413496"} Jun 10 13:00:40.397303 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.397233 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vk784" event={"ID":"7c46c69f-3d3c-408f-a7b6-ba558717e019","Type":"ContainerStarted","Data":"2cd6eb808683fa7f0b5ecd4e55f34a5edf71b27b4206e4ae9f8b07c8da968e07"} Jun 10 13:00:40.397371 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.397355 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vk784" Jun 10 13:00:40.397578 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:40.397558 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l9wqp\" is forbidden: User \"system:node:ip-10-0-128-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-230.ec2.internal' and this object" podUID="290f9654-ba81-4d5f-bf63-9ff58d0a408b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:40.402299 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.402282 2581 scope.go:117] "RemoveContainer" containerID="32ac4661941ab9d244c5ecbb665f0d9d61b2ea69846b69b5a31443981f37af77" Jun 10 13:00:40.402545 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:40.402524 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32ac4661941ab9d244c5ecbb665f0d9d61b2ea69846b69b5a31443981f37af77\": container with ID starting with 32ac4661941ab9d244c5ecbb665f0d9d61b2ea69846b69b5a31443981f37af77 not found: ID does not exist" containerID="32ac4661941ab9d244c5ecbb665f0d9d61b2ea69846b69b5a31443981f37af77" Jun 10 13:00:40.402602 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.402558 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32ac4661941ab9d244c5ecbb665f0d9d61b2ea69846b69b5a31443981f37af77"} err="failed to get container status \"32ac4661941ab9d244c5ecbb665f0d9d61b2ea69846b69b5a31443981f37af77\": rpc error: code = NotFound desc = could not find container \"32ac4661941ab9d244c5ecbb665f0d9d61b2ea69846b69b5a31443981f37af77\": container with ID starting with 32ac4661941ab9d244c5ecbb665f0d9d61b2ea69846b69b5a31443981f37af77 not found: ID does not exist" Jun 10 13:00:40.402602 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.402582 2581 scope.go:117] "RemoveContainer" containerID="bae76b58875c30ac7b005444a50d8df443fa2afb7a94287fde90ad90566d9ab0" Jun 10 13:00:40.410615 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.410604 2581 scope.go:117] "RemoveContainer" containerID="bae76b58875c30ac7b005444a50d8df443fa2afb7a94287fde90ad90566d9ab0" Jun 10 13:00:40.410850 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:40.410834 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bae76b58875c30ac7b005444a50d8df443fa2afb7a94287fde90ad90566d9ab0\": container with ID starting with bae76b58875c30ac7b005444a50d8df443fa2afb7a94287fde90ad90566d9ab0 not found: ID does not exist" containerID="bae76b58875c30ac7b005444a50d8df443fa2afb7a94287fde90ad90566d9ab0" Jun 10 13:00:40.410891 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.410856 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bae76b58875c30ac7b005444a50d8df443fa2afb7a94287fde90ad90566d9ab0"} err="failed to get container status \"bae76b58875c30ac7b005444a50d8df443fa2afb7a94287fde90ad90566d9ab0\": rpc error: code = NotFound desc = could not find container \"bae76b58875c30ac7b005444a50d8df443fa2afb7a94287fde90ad90566d9ab0\": container with ID starting with bae76b58875c30ac7b005444a50d8df443fa2afb7a94287fde90ad90566d9ab0 not found: ID does not exist" Jun 10 13:00:40.420893 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.420850 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vk784" podStartSLOduration=1.420834626 podStartE2EDuration="1.420834626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:00:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:00:40.419996002 +0000 UTC m=+702.160210480" watchObservedRunningTime="2026-06-10 13:00:40.420834626 +0000 UTC m=+702.161049109" Jun 10 13:00:40.449100 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.449051 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr" podStartSLOduration=1.449034117 podStartE2EDuration="1.449034117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:00:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:00:40.448608479 +0000 UTC m=+702.188822960" watchObservedRunningTime="2026-06-10 13:00:40.449034117 +0000 UTC m=+702.189248599" Jun 10 13:00:40.451732 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:40.451705 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-srktq\" is forbidden: User \"system:node:ip-10-0-128-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-230.ec2.internal' and this object" podUID="72e8ec98-9362-407e-9030-0a4284c9c1ca" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" Jun 10 13:00:40.454099 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:40.454079 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l9wqp\" is forbidden: User \"system:node:ip-10-0-128-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-230.ec2.internal' and this object" podUID="290f9654-ba81-4d5f-bf63-9ff58d0a408b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:40.460672 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:40.460649 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-srktq\" is forbidden: User \"system:node:ip-10-0-128-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-230.ec2.internal' and this object" podUID="72e8ec98-9362-407e-9030-0a4284c9c1ca" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-srktq" Jun 10 13:00:40.463032 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:40.463009 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l9wqp\" is forbidden: User \"system:node:ip-10-0-128-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-230.ec2.internal' and this object" podUID="290f9654-ba81-4d5f-bf63-9ff58d0a408b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l9wqp" Jun 10 13:00:40.463117 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.463068 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvrhj\" (UniqueName: \"kubernetes.io/projected/4bc87560-33a3-4edc-aa18-abe71940a3e5-kube-api-access-nvrhj\") pod \"kuadrant-operator-controller-manager-86878c769f-knnrb\" (UID: \"4bc87560-33a3-4edc-aa18-abe71940a3e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb" Jun 10 13:00:40.463224 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.463205 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4bc87560-33a3-4edc-aa18-abe71940a3e5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-knnrb\" (UID: \"4bc87560-33a3-4edc-aa18-abe71940a3e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb" Jun 10 13:00:40.564434 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.564406 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvrhj\" (UniqueName: \"kubernetes.io/projected/4bc87560-33a3-4edc-aa18-abe71940a3e5-kube-api-access-nvrhj\") pod \"kuadrant-operator-controller-manager-86878c769f-knnrb\" (UID: \"4bc87560-33a3-4edc-aa18-abe71940a3e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb" Jun 10 13:00:40.564584 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.564485 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4bc87560-33a3-4edc-aa18-abe71940a3e5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-knnrb\" (UID: \"4bc87560-33a3-4edc-aa18-abe71940a3e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb" Jun 10 13:00:40.564862 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.564843 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4bc87560-33a3-4edc-aa18-abe71940a3e5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-knnrb\" (UID: \"4bc87560-33a3-4edc-aa18-abe71940a3e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb" Jun 10 13:00:40.589235 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.589162 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvrhj\" (UniqueName: \"kubernetes.io/projected/4bc87560-33a3-4edc-aa18-abe71940a3e5-kube-api-access-nvrhj\") pod \"kuadrant-operator-controller-manager-86878c769f-knnrb\" (UID: \"4bc87560-33a3-4edc-aa18-abe71940a3e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb" Jun 10 13:00:40.642315 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.642287 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb" Jun 10 13:00:40.722073 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.722039 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="290f9654-ba81-4d5f-bf63-9ff58d0a408b" path="/var/lib/kubelet/pods/290f9654-ba81-4d5f-bf63-9ff58d0a408b/volumes" Jun 10 13:00:40.722402 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.722388 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72e8ec98-9362-407e-9030-0a4284c9c1ca" path="/var/lib/kubelet/pods/72e8ec98-9362-407e-9030-0a4284c9c1ca/volumes" Jun 10 13:00:40.790453 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:40.790420 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb"] Jun 10 13:00:40.793641 ip-10-0-128-230 kubenswrapper[2581]: W0610 13:00:40.793613 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bc87560_33a3_4edc_aa18_abe71940a3e5.slice/crio-6e75776c9eea635030c11e1ac7cbbd7519d465712d4c96bf9139293fe2af9471 WatchSource:0}: Error finding container 6e75776c9eea635030c11e1ac7cbbd7519d465712d4c96bf9139293fe2af9471: Status 404 returned error can't find the container with id 6e75776c9eea635030c11e1ac7cbbd7519d465712d4c96bf9139293fe2af9471 Jun 10 13:00:41.406211 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:41.406157 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb" event={"ID":"4bc87560-33a3-4edc-aa18-abe71940a3e5","Type":"ContainerStarted","Data":"17d4d042256994a905df9457904a97ec5a8933f0bc36b9b0c5ea416d1601c3b4"} Jun 10 13:00:41.406364 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:41.406227 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb" event={"ID":"4bc87560-33a3-4edc-aa18-abe71940a3e5","Type":"ContainerStarted","Data":"6e75776c9eea635030c11e1ac7cbbd7519d465712d4c96bf9139293fe2af9471"} Jun 10 13:00:41.406364 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:41.406289 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb" Jun 10 13:00:41.427855 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:41.427798 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb" podStartSLOduration=1.427783126 podStartE2EDuration="1.427783126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:00:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:00:41.427495122 +0000 UTC m=+703.167709604" watchObservedRunningTime="2026-06-10 13:00:41.427783126 +0000 UTC m=+703.167997606" Jun 10 13:00:45.105408 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:45.105370 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:45.205108 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:45.205074 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:47.715799 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:47.715712 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:47.815082 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:47.815034 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:51.408874 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:51.408835 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr" Jun 10 13:00:51.409271 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:51.408899 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vk784" Jun 10 13:00:51.542200 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:51.542132 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:51.643799 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:51.643764 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:52.414416 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:52.414389 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb" Jun 10 13:00:52.495802 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:52.495766 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr"] Jun 10 13:00:52.495981 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:52.495939 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr" podUID="e8c4e12e-abfc-4be7-9b75-41c975e823bd" containerName="manager" containerID="cri-o://0b8105e61e9e5c7da263bde9f0da030c39a728dfd4c8d8354cf4725a593b682d" gracePeriod=10 Jun 10 13:00:52.746577 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:52.746554 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr" Jun 10 13:00:52.875332 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:52.875306 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8c4e12e-abfc-4be7-9b75-41c975e823bd-extensions-socket-volume\") pod \"e8c4e12e-abfc-4be7-9b75-41c975e823bd\" (UID: \"e8c4e12e-abfc-4be7-9b75-41c975e823bd\") " Jun 10 13:00:52.875516 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:52.875369 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbrnh\" (UniqueName: \"kubernetes.io/projected/e8c4e12e-abfc-4be7-9b75-41c975e823bd-kube-api-access-hbrnh\") pod \"e8c4e12e-abfc-4be7-9b75-41c975e823bd\" (UID: \"e8c4e12e-abfc-4be7-9b75-41c975e823bd\") " Jun 10 13:00:52.875787 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:52.875755 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8c4e12e-abfc-4be7-9b75-41c975e823bd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e8c4e12e-abfc-4be7-9b75-41c975e823bd" (UID: "e8c4e12e-abfc-4be7-9b75-41c975e823bd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 13:00:52.877611 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:52.877589 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8c4e12e-abfc-4be7-9b75-41c975e823bd-kube-api-access-hbrnh" (OuterVolumeSpecName: "kube-api-access-hbrnh") pod "e8c4e12e-abfc-4be7-9b75-41c975e823bd" (UID: "e8c4e12e-abfc-4be7-9b75-41c975e823bd"). InnerVolumeSpecName "kube-api-access-hbrnh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:00:52.976372 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:52.976296 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8c4e12e-abfc-4be7-9b75-41c975e823bd-extensions-socket-volume\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:00:52.976372 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:52.976320 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbrnh\" (UniqueName: \"kubernetes.io/projected/e8c4e12e-abfc-4be7-9b75-41c975e823bd-kube-api-access-hbrnh\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:00:53.473385 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:53.473348 2581 generic.go:356] "Generic (PLEG): container finished" podID="e8c4e12e-abfc-4be7-9b75-41c975e823bd" containerID="0b8105e61e9e5c7da263bde9f0da030c39a728dfd4c8d8354cf4725a593b682d" exitCode=0 Jun 10 13:00:53.473781 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:53.473394 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr" event={"ID":"e8c4e12e-abfc-4be7-9b75-41c975e823bd","Type":"ContainerDied","Data":"0b8105e61e9e5c7da263bde9f0da030c39a728dfd4c8d8354cf4725a593b682d"} Jun 10 13:00:53.473781 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:53.473401 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr" Jun 10 13:00:53.473781 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:53.473446 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr" event={"ID":"e8c4e12e-abfc-4be7-9b75-41c975e823bd","Type":"ContainerDied","Data":"6fab53435ded79c807f6b7b06d019f61bb78ea99f8e7914d865be25f0ef92927"} Jun 10 13:00:53.473781 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:53.473477 2581 scope.go:117] "RemoveContainer" containerID="0b8105e61e9e5c7da263bde9f0da030c39a728dfd4c8d8354cf4725a593b682d" Jun 10 13:00:53.482967 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:53.482948 2581 scope.go:117] "RemoveContainer" containerID="0b8105e61e9e5c7da263bde9f0da030c39a728dfd4c8d8354cf4725a593b682d" Jun 10 13:00:53.483298 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:00:53.483273 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b8105e61e9e5c7da263bde9f0da030c39a728dfd4c8d8354cf4725a593b682d\": container with ID starting with 0b8105e61e9e5c7da263bde9f0da030c39a728dfd4c8d8354cf4725a593b682d not found: ID does not exist" containerID="0b8105e61e9e5c7da263bde9f0da030c39a728dfd4c8d8354cf4725a593b682d" Jun 10 13:00:53.483404 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:53.483305 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b8105e61e9e5c7da263bde9f0da030c39a728dfd4c8d8354cf4725a593b682d"} err="failed to get container status \"0b8105e61e9e5c7da263bde9f0da030c39a728dfd4c8d8354cf4725a593b682d\": rpc error: code = NotFound desc = could not find container \"0b8105e61e9e5c7da263bde9f0da030c39a728dfd4c8d8354cf4725a593b682d\": container with ID starting with 0b8105e61e9e5c7da263bde9f0da030c39a728dfd4c8d8354cf4725a593b682d not found: ID does not exist" Jun 10 13:00:53.533305 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:53.533277 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr"] Jun 10 13:00:53.555795 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:53.555763 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hd5gr"] Jun 10 13:00:54.718223 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:54.718189 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8c4e12e-abfc-4be7-9b75-41c975e823bd" path="/var/lib/kubelet/pods/e8c4e12e-abfc-4be7-9b75-41c975e823bd/volumes" Jun 10 13:00:56.747080 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:56.747042 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:56.847598 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:56.847555 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:00:58.725561 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:58.725532 2581 scope.go:117] "RemoveContainer" containerID="81926b94756d490d9a5e987a104a0be50482e8fb2839eeff21bb59680e4effc3" Jun 10 13:00:58.733998 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:58.733973 2581 scope.go:117] "RemoveContainer" containerID="48b470d8405ce491abe7b7aa4aab7282dec1752b180803a8316b2d968c509acd" Jun 10 13:00:58.741885 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:58.741868 2581 scope.go:117] "RemoveContainer" containerID="ac6f96d9f13124f1ac722cc92a6bac12de372ac01a2c00890275f9753d47658b" Jun 10 13:00:58.749805 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:58.749789 2581 scope.go:117] "RemoveContainer" containerID="6d131879f5b77ee8527b1a4aaed44ec16963196e181d6f51e5afa0f9995b24aa" Jun 10 13:00:58.756602 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:58.756577 2581 scope.go:117] "RemoveContainer" containerID="4bd8f67fea5141907df5cc5db250ed8b78fb73575d57ff1c9dbe78fb4cf9d991" Jun 10 13:00:58.764005 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:58.763987 2581 scope.go:117] "RemoveContainer" containerID="d944d83475b0c76b297f046ff6133ae141b071dc39e71240a5c43927d5efc6bd" Jun 10 13:00:58.771058 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:00:58.771039 2581 scope.go:117] "RemoveContainer" containerID="96c7f4a498640963ae03094ed3cebc9144e7103d6872b1d9eee9fc5f08483eb4" Jun 10 13:01:00.506608 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:00.506573 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:01:00.607686 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:00.607654 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:01:01.695296 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:01.695258 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:01:01.760800 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:01.760745 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:01:03.969809 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:03.969768 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:01:04.054686 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:04.054656 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:01:05.929439 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:05.929390 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:01:06.030131 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:06.030101 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:01:08.899640 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:08.899606 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm"] Jun 10 13:01:08.900125 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:08.900103 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8c4e12e-abfc-4be7-9b75-41c975e823bd" containerName="manager" Jun 10 13:01:08.900125 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:08.900126 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8c4e12e-abfc-4be7-9b75-41c975e823bd" containerName="manager" Jun 10 13:01:08.900288 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:08.900273 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8c4e12e-abfc-4be7-9b75-41c975e823bd" containerName="manager" Jun 10 13:01:08.903780 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:08.903763 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:08.906685 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:08.906662 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-kb66s\"" Jun 10 13:01:08.916355 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:08.916336 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm"] Jun 10 13:01:08.929495 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:08.929449 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:01:09.023389 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.023361 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a97a42-4ec9-4eea-a743-9baddfd1f118-credential-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.023560 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.023405 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7a97a42-4ec9-4eea-a743-9baddfd1f118-istio-token\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.023560 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.023428 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7a97a42-4ec9-4eea-a743-9baddfd1f118-istio-envoy\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.023560 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.023505 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a97a42-4ec9-4eea-a743-9baddfd1f118-workload-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.023687 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.023566 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7a97a42-4ec9-4eea-a743-9baddfd1f118-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.023687 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.023626 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7a97a42-4ec9-4eea-a743-9baddfd1f118-istio-data\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.023687 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.023654 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7a97a42-4ec9-4eea-a743-9baddfd1f118-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.023687 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.023670 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsw6g\" (UniqueName: \"kubernetes.io/projected/d7a97a42-4ec9-4eea-a743-9baddfd1f118-kube-api-access-bsw6g\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.023822 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.023690 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7a97a42-4ec9-4eea-a743-9baddfd1f118-workload-certs\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.029497 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.029471 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:01:09.124770 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.124739 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7a97a42-4ec9-4eea-a743-9baddfd1f118-istio-token\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.124770 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.124772 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7a97a42-4ec9-4eea-a743-9baddfd1f118-istio-envoy\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.124983 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.124801 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a97a42-4ec9-4eea-a743-9baddfd1f118-workload-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.124983 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.124839 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7a97a42-4ec9-4eea-a743-9baddfd1f118-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.124983 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.124894 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7a97a42-4ec9-4eea-a743-9baddfd1f118-istio-data\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.124983 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.124934 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7a97a42-4ec9-4eea-a743-9baddfd1f118-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.124983 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.124956 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsw6g\" (UniqueName: \"kubernetes.io/projected/d7a97a42-4ec9-4eea-a743-9baddfd1f118-kube-api-access-bsw6g\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.124983 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.124982 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7a97a42-4ec9-4eea-a743-9baddfd1f118-workload-certs\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.125702 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.125010 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a97a42-4ec9-4eea-a743-9baddfd1f118-credential-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.125702 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.125396 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a97a42-4ec9-4eea-a743-9baddfd1f118-workload-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.125702 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.125456 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7a97a42-4ec9-4eea-a743-9baddfd1f118-istio-data\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.125702 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.125600 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7a97a42-4ec9-4eea-a743-9baddfd1f118-workload-certs\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.125702 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.125612 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7a97a42-4ec9-4eea-a743-9baddfd1f118-credential-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.125702 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.125679 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7a97a42-4ec9-4eea-a743-9baddfd1f118-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.127269 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.127245 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7a97a42-4ec9-4eea-a743-9baddfd1f118-istio-envoy\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.127501 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.127484 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7a97a42-4ec9-4eea-a743-9baddfd1f118-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.134270 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.134244 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7a97a42-4ec9-4eea-a743-9baddfd1f118-istio-token\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.134452 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.134434 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsw6g\" (UniqueName: \"kubernetes.io/projected/d7a97a42-4ec9-4eea-a743-9baddfd1f118-kube-api-access-bsw6g\") pod \"maas-default-gateway-openshift-default-8559cd5744-4wqlm\" (UID: \"d7a97a42-4ec9-4eea-a743-9baddfd1f118\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.214560 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.214411 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:09.351871 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.351817 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm"] Jun 10 13:01:09.354111 ip-10-0-128-230 kubenswrapper[2581]: W0610 13:01:09.354081 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7a97a42_4ec9_4eea_a743_9baddfd1f118.slice/crio-d2ad25680c7e57ec30fe3776e0f9f9bbcdfa909d0fe81d0bdfae639cfe51b157 WatchSource:0}: Error finding container d2ad25680c7e57ec30fe3776e0f9f9bbcdfa909d0fe81d0bdfae639cfe51b157: Status 404 returned error can't find the container with id d2ad25680c7e57ec30fe3776e0f9f9bbcdfa909d0fe81d0bdfae639cfe51b157 Jun 10 13:01:09.356205 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.356150 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jun 10 13:01:09.356320 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.356253 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jun 10 13:01:09.356320 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.356297 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jun 10 13:01:09.554603 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.554501 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" event={"ID":"d7a97a42-4ec9-4eea-a743-9baddfd1f118","Type":"ContainerStarted","Data":"cf5f9fc43c2b3c44268998b3a30648ed43e4c5475ab034b94768a81a5a9f9f5c"} Jun 10 13:01:09.554603 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.554556 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" event={"ID":"d7a97a42-4ec9-4eea-a743-9baddfd1f118","Type":"ContainerStarted","Data":"d2ad25680c7e57ec30fe3776e0f9f9bbcdfa909d0fe81d0bdfae639cfe51b157"} Jun 10 13:01:09.577926 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:09.577872 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" podStartSLOduration=1.5778564940000002 podStartE2EDuration="1.577856494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:01:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:01:09.576756302 +0000 UTC m=+731.316970784" watchObservedRunningTime="2026-06-10 13:01:09.577856494 +0000 UTC m=+731.318070978" Jun 10 13:01:10.214782 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:10.214744 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:10.219222 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:10.219194 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:10.559431 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:10.559348 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:10.560215 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:10.560194 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:01:13.201148 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:13.201119 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:01:13.708140 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:13.708102 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:01:13.716423 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:13.716396 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s7r5z"] Jun 10 13:01:13.718725 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:13.718709 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-s7r5z" Jun 10 13:01:13.721331 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:13.721313 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 10 13:01:13.721436 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:13.721371 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jm4pz\"" Jun 10 13:01:13.732590 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:13.732570 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s7r5z"] Jun 10 13:01:13.816445 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:13.816400 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s7r5z"] Jun 10 13:01:13.820662 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:13.820629 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:01:13.871607 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:13.871573 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3ebfb32-e6bd-4457-a82c-96d8d485095a-config-file\") pod \"limitador-limitador-64c94fd64f-s7r5z\" (UID: \"f3ebfb32-e6bd-4457-a82c-96d8d485095a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s7r5z" Jun 10 13:01:13.871780 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:13.871684 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4drt\" (UniqueName: \"kubernetes.io/projected/f3ebfb32-e6bd-4457-a82c-96d8d485095a-kube-api-access-n4drt\") pod \"limitador-limitador-64c94fd64f-s7r5z\" (UID: \"f3ebfb32-e6bd-4457-a82c-96d8d485095a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s7r5z" Jun 10 13:01:13.880961 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:13.880930 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:01:13.972331 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:13.972252 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4drt\" (UniqueName: \"kubernetes.io/projected/f3ebfb32-e6bd-4457-a82c-96d8d485095a-kube-api-access-n4drt\") pod \"limitador-limitador-64c94fd64f-s7r5z\" (UID: \"f3ebfb32-e6bd-4457-a82c-96d8d485095a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s7r5z" Jun 10 13:01:13.972331 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:13.972306 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3ebfb32-e6bd-4457-a82c-96d8d485095a-config-file\") pod \"limitador-limitador-64c94fd64f-s7r5z\" (UID: \"f3ebfb32-e6bd-4457-a82c-96d8d485095a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s7r5z" Jun 10 13:01:13.972838 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:13.972819 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3ebfb32-e6bd-4457-a82c-96d8d485095a-config-file\") pod \"limitador-limitador-64c94fd64f-s7r5z\" (UID: \"f3ebfb32-e6bd-4457-a82c-96d8d485095a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s7r5z" Jun 10 13:01:13.983604 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:13.983578 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4drt\" (UniqueName: \"kubernetes.io/projected/f3ebfb32-e6bd-4457-a82c-96d8d485095a-kube-api-access-n4drt\") pod \"limitador-limitador-64c94fd64f-s7r5z\" (UID: \"f3ebfb32-e6bd-4457-a82c-96d8d485095a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s7r5z" Jun 10 13:01:14.029394 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:14.029366 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-s7r5z" Jun 10 13:01:14.164714 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:14.164692 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s7r5z"] Jun 10 13:01:14.166579 ip-10-0-128-230 kubenswrapper[2581]: W0610 13:01:14.166551 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3ebfb32_e6bd_4457_a82c_96d8d485095a.slice/crio-302528ea32338b07d814180bc9a448f21e08ee4c247a630087f32439f65c9803 WatchSource:0}: Error finding container 302528ea32338b07d814180bc9a448f21e08ee4c247a630087f32439f65c9803: Status 404 returned error can't find the container with id 302528ea32338b07d814180bc9a448f21e08ee4c247a630087f32439f65c9803 Jun 10 13:01:14.580655 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:14.580620 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-s7r5z" event={"ID":"f3ebfb32-e6bd-4457-a82c-96d8d485095a","Type":"ContainerStarted","Data":"302528ea32338b07d814180bc9a448f21e08ee4c247a630087f32439f65c9803"} Jun 10 13:01:14.618689 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:14.618647 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:01:14.719129 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:14.719098 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:01:14.752093 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:14.752059 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-cgx62"] Jun 10 13:01:14.754996 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:14.754980 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-cgx62" Jun 10 13:01:14.757632 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:14.757610 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-n5gh5\"" Jun 10 13:01:14.761290 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:14.761268 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-cgx62"] Jun 10 13:01:14.879943 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:14.879903 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngn9b\" (UniqueName: \"kubernetes.io/projected/25c43eaf-b062-484d-9991-a8d724c77902-kube-api-access-ngn9b\") pod \"authorino-75d556bd77-cgx62\" (UID: \"25c43eaf-b062-484d-9991-a8d724c77902\") " pod="kuadrant-system/authorino-75d556bd77-cgx62" Jun 10 13:01:14.981735 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:14.981150 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngn9b\" (UniqueName: \"kubernetes.io/projected/25c43eaf-b062-484d-9991-a8d724c77902-kube-api-access-ngn9b\") pod \"authorino-75d556bd77-cgx62\" (UID: \"25c43eaf-b062-484d-9991-a8d724c77902\") " pod="kuadrant-system/authorino-75d556bd77-cgx62" Jun 10 13:01:14.993191 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:14.993137 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngn9b\" (UniqueName: \"kubernetes.io/projected/25c43eaf-b062-484d-9991-a8d724c77902-kube-api-access-ngn9b\") pod \"authorino-75d556bd77-cgx62\" (UID: \"25c43eaf-b062-484d-9991-a8d724c77902\") " pod="kuadrant-system/authorino-75d556bd77-cgx62" Jun 10 13:01:15.077680 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:15.077411 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-cgx62" Jun 10 13:01:15.255109 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:15.255078 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-cgx62"] Jun 10 13:01:15.257724 ip-10-0-128-230 kubenswrapper[2581]: W0610 13:01:15.257687 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25c43eaf_b062_484d_9991_a8d724c77902.slice/crio-aecafb194aa53d4163bb74cdacaf93183126d72f7f44466de1e48fdd8e1b445c WatchSource:0}: Error finding container aecafb194aa53d4163bb74cdacaf93183126d72f7f44466de1e48fdd8e1b445c: Status 404 returned error can't find the container with id aecafb194aa53d4163bb74cdacaf93183126d72f7f44466de1e48fdd8e1b445c Jun 10 13:01:15.588006 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:15.587926 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-cgx62" event={"ID":"25c43eaf-b062-484d-9991-a8d724c77902","Type":"ContainerStarted","Data":"aecafb194aa53d4163bb74cdacaf93183126d72f7f44466de1e48fdd8e1b445c"} Jun 10 13:01:17.604393 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:17.604350 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-s7r5z" event={"ID":"f3ebfb32-e6bd-4457-a82c-96d8d485095a","Type":"ContainerStarted","Data":"0c97f49312dd82dedb85d4267805a42a4a7094cff5afd64128a19741546c343a"} Jun 10 13:01:17.604875 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:17.604471 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-s7r5z" Jun 10 13:01:17.627002 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:17.626940 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-s7r5z" podStartSLOduration=1.783441542 podStartE2EDuration="4.626917793s" totalImagesPullingTime="2.843476251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:01:17.62419455 +0000 UTC m=+739.364409026" watchObservedRunningTime="2026-06-10 13:01:17.626917793 +0000 UTC m=+739.367132277" Jun 10 13:01:19.616609 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:19.616572 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-cgx62" event={"ID":"25c43eaf-b062-484d-9991-a8d724c77902","Type":"ContainerStarted","Data":"8b1a4ba3f6badda783a9747e0e98d54cb1404e69acde9de0fa90f9364c1c8533"} Jun 10 13:01:19.634073 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:19.634019 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-cgx62" podStartSLOduration=1.884711746 podStartE2EDuration="5.63400187s" totalImagesPullingTime="3.749290124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:01:19.633922399 +0000 UTC m=+741.374136881" watchObservedRunningTime="2026-06-10 13:01:19.63400187 +0000 UTC m=+741.374216352" Jun 10 13:01:28.612216 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:28.612162 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-s7r5z" Jun 10 13:01:29.009971 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:29.009892 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s7r5z"] Jun 10 13:01:29.010134 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:29.010073 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-s7r5z" podUID="f3ebfb32-e6bd-4457-a82c-96d8d485095a" containerName="limitador" containerID="cri-o://0c97f49312dd82dedb85d4267805a42a4a7094cff5afd64128a19741546c343a" gracePeriod=30 Jun 10 13:01:29.550908 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:29.550885 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-s7r5z" Jun 10 13:01:29.616182 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:29.616140 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3ebfb32-e6bd-4457-a82c-96d8d485095a-config-file\") pod \"f3ebfb32-e6bd-4457-a82c-96d8d485095a\" (UID: \"f3ebfb32-e6bd-4457-a82c-96d8d485095a\") " Jun 10 13:01:29.616521 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:29.616200 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4drt\" (UniqueName: \"kubernetes.io/projected/f3ebfb32-e6bd-4457-a82c-96d8d485095a-kube-api-access-n4drt\") pod \"f3ebfb32-e6bd-4457-a82c-96d8d485095a\" (UID: \"f3ebfb32-e6bd-4457-a82c-96d8d485095a\") " Jun 10 13:01:29.616521 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:29.616454 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3ebfb32-e6bd-4457-a82c-96d8d485095a-config-file" (OuterVolumeSpecName: "config-file") pod "f3ebfb32-e6bd-4457-a82c-96d8d485095a" (UID: "f3ebfb32-e6bd-4457-a82c-96d8d485095a"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 13:01:29.618307 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:29.618281 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3ebfb32-e6bd-4457-a82c-96d8d485095a-kube-api-access-n4drt" (OuterVolumeSpecName: "kube-api-access-n4drt") pod "f3ebfb32-e6bd-4457-a82c-96d8d485095a" (UID: "f3ebfb32-e6bd-4457-a82c-96d8d485095a"). InnerVolumeSpecName "kube-api-access-n4drt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:01:29.665768 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:29.665745 2581 generic.go:356] "Generic (PLEG): container finished" podID="f3ebfb32-e6bd-4457-a82c-96d8d485095a" containerID="0c97f49312dd82dedb85d4267805a42a4a7094cff5afd64128a19741546c343a" exitCode=0 Jun 10 13:01:29.665891 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:29.665781 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-s7r5z" event={"ID":"f3ebfb32-e6bd-4457-a82c-96d8d485095a","Type":"ContainerDied","Data":"0c97f49312dd82dedb85d4267805a42a4a7094cff5afd64128a19741546c343a"} Jun 10 13:01:29.665891 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:29.665787 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-s7r5z" Jun 10 13:01:29.665891 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:29.665819 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-s7r5z" event={"ID":"f3ebfb32-e6bd-4457-a82c-96d8d485095a","Type":"ContainerDied","Data":"302528ea32338b07d814180bc9a448f21e08ee4c247a630087f32439f65c9803"} Jun 10 13:01:29.665891 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:29.665841 2581 scope.go:117] "RemoveContainer" containerID="0c97f49312dd82dedb85d4267805a42a4a7094cff5afd64128a19741546c343a" Jun 10 13:01:29.674766 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:29.674751 2581 scope.go:117] "RemoveContainer" containerID="0c97f49312dd82dedb85d4267805a42a4a7094cff5afd64128a19741546c343a" Jun 10 13:01:29.675005 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:01:29.674987 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c97f49312dd82dedb85d4267805a42a4a7094cff5afd64128a19741546c343a\": container with ID starting with 0c97f49312dd82dedb85d4267805a42a4a7094cff5afd64128a19741546c343a not found: ID does not exist" containerID="0c97f49312dd82dedb85d4267805a42a4a7094cff5afd64128a19741546c343a" Jun 10 13:01:29.675053 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:29.675014 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c97f49312dd82dedb85d4267805a42a4a7094cff5afd64128a19741546c343a"} err="failed to get container status \"0c97f49312dd82dedb85d4267805a42a4a7094cff5afd64128a19741546c343a\": rpc error: code = NotFound desc = could not find container \"0c97f49312dd82dedb85d4267805a42a4a7094cff5afd64128a19741546c343a\": container with ID starting with 0c97f49312dd82dedb85d4267805a42a4a7094cff5afd64128a19741546c343a not found: ID does not exist" Jun 10 13:01:29.689583 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:29.689561 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s7r5z"] Jun 10 13:01:29.694179 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:29.694146 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s7r5z"] Jun 10 13:01:29.716794 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:29.716772 2581 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3ebfb32-e6bd-4457-a82c-96d8d485095a-config-file\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:01:29.716794 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:29.716792 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4drt\" (UniqueName: \"kubernetes.io/projected/f3ebfb32-e6bd-4457-a82c-96d8d485095a-kube-api-access-n4drt\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:01:30.718180 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:30.718134 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3ebfb32-e6bd-4457-a82c-96d8d485095a" path="/var/lib/kubelet/pods/f3ebfb32-e6bd-4457-a82c-96d8d485095a/volumes" Jun 10 13:01:34.805513 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:34.805478 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:01:46.948415 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:46.948375 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:01:47.062448 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:47.062401 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:01:47.292372 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:47.292277 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:01:55.804073 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:55.804042 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-nbvvx"] Jun 10 13:01:55.804534 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:55.804422 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3ebfb32-e6bd-4457-a82c-96d8d485095a" containerName="limitador" Jun 10 13:01:55.804534 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:55.804434 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3ebfb32-e6bd-4457-a82c-96d8d485095a" containerName="limitador" Jun 10 13:01:55.804534 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:55.804494 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3ebfb32-e6bd-4457-a82c-96d8d485095a" containerName="limitador" Jun 10 13:01:55.811063 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:55.811045 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-d67dbbb48-nbvvx" Jun 10 13:01:55.815214 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:55.815188 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 10 13:01:55.815407 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:55.815192 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"keycloak-operator-dockercfg-vsxk8\"" Jun 10 13:01:55.815407 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:55.815193 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 10 13:01:55.816285 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:55.816266 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-nbvvx"] Jun 10 13:01:55.951044 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:55.951009 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wtgk\" (UniqueName: \"kubernetes.io/projected/4deac7e5-6dd1-4439-8314-51261e9453f7-kube-api-access-8wtgk\") pod \"keycloak-operator-d67dbbb48-nbvvx\" (UID: \"4deac7e5-6dd1-4439-8314-51261e9453f7\") " pod="keycloak-system/keycloak-operator-d67dbbb48-nbvvx" Jun 10 13:01:56.051552 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:56.051519 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wtgk\" (UniqueName: \"kubernetes.io/projected/4deac7e5-6dd1-4439-8314-51261e9453f7-kube-api-access-8wtgk\") pod \"keycloak-operator-d67dbbb48-nbvvx\" (UID: \"4deac7e5-6dd1-4439-8314-51261e9453f7\") " pod="keycloak-system/keycloak-operator-d67dbbb48-nbvvx" Jun 10 13:01:56.070200 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:56.070114 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wtgk\" (UniqueName: \"kubernetes.io/projected/4deac7e5-6dd1-4439-8314-51261e9453f7-kube-api-access-8wtgk\") pod \"keycloak-operator-d67dbbb48-nbvvx\" (UID: \"4deac7e5-6dd1-4439-8314-51261e9453f7\") " pod="keycloak-system/keycloak-operator-d67dbbb48-nbvvx" Jun 10 13:01:56.122020 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:56.121987 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-d67dbbb48-nbvvx" Jun 10 13:01:56.256747 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:56.256723 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-nbvvx"] Jun 10 13:01:56.258894 ip-10-0-128-230 kubenswrapper[2581]: W0610 13:01:56.258867 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4deac7e5_6dd1_4439_8314_51261e9453f7.slice/crio-3b7f4db4bddbff7c731da5c3a71dd651c8d1481920422965c12ebe90822363ad WatchSource:0}: Error finding container 3b7f4db4bddbff7c731da5c3a71dd651c8d1481920422965c12ebe90822363ad: Status 404 returned error can't find the container with id 3b7f4db4bddbff7c731da5c3a71dd651c8d1481920422965c12ebe90822363ad Jun 10 13:01:56.792162 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:01:56.792131 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-d67dbbb48-nbvvx" event={"ID":"4deac7e5-6dd1-4439-8314-51261e9453f7","Type":"ContainerStarted","Data":"3b7f4db4bddbff7c731da5c3a71dd651c8d1481920422965c12ebe90822363ad"} Jun 10 13:02:02.824559 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:02.824469 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-d67dbbb48-nbvvx" event={"ID":"4deac7e5-6dd1-4439-8314-51261e9453f7","Type":"ContainerStarted","Data":"98bd4854497f9d1649ea2e97d1edc6f2b0568ec41d92da70f67a8d4b65743cdc"} Jun 10 13:02:02.848094 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:02.848040 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/keycloak-operator-d67dbbb48-nbvvx" podStartSLOduration=1.652374815 podStartE2EDuration="7.84802469s" totalImagesPullingTime="6.195649875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:01:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:02:02.845792745 +0000 UTC m=+784.586007227" watchObservedRunningTime="2026-06-10 13:02:02.84802469 +0000 UTC m=+784.588239171" Jun 10 13:02:11.301751 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:11.301716 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:02:11.377089 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:11.377050 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:02:18.100299 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:18.100196 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:02:41.753031 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:41.752988 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:02:41.971199 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:41.971150 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jbwpl"] Jun 10 13:02:41.980307 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:41.980280 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jbwpl" Jun 10 13:02:41.983789 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:41.983764 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jbwpl"] Jun 10 13:02:42.049782 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:42.049701 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:02:42.086286 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:42.086257 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxthd\" (UniqueName: \"kubernetes.io/projected/f54a2157-e72a-45ec-9463-75befe8fd328-kube-api-access-vxthd\") pod \"authorino-5d4b67c446-jbwpl\" (UID: \"f54a2157-e72a-45ec-9463-75befe8fd328\") " pod="kuadrant-system/authorino-5d4b67c446-jbwpl" Jun 10 13:02:42.181280 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:42.181248 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jbwpl"] Jun 10 13:02:42.181534 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:02:42.181513 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-vxthd], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-jbwpl" podUID="f54a2157-e72a-45ec-9463-75befe8fd328" Jun 10 13:02:42.187629 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:42.187603 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxthd\" (UniqueName: \"kubernetes.io/projected/f54a2157-e72a-45ec-9463-75befe8fd328-kube-api-access-vxthd\") pod \"authorino-5d4b67c446-jbwpl\" (UID: \"f54a2157-e72a-45ec-9463-75befe8fd328\") " pod="kuadrant-system/authorino-5d4b67c446-jbwpl" Jun 10 13:02:42.196640 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:42.196613 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxthd\" (UniqueName: \"kubernetes.io/projected/f54a2157-e72a-45ec-9463-75befe8fd328-kube-api-access-vxthd\") pod \"authorino-5d4b67c446-jbwpl\" (UID: \"f54a2157-e72a-45ec-9463-75befe8fd328\") " pod="kuadrant-system/authorino-5d4b67c446-jbwpl" Jun 10 13:02:42.220232 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:42.220200 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7f4c9f7664-s8rkz"] Jun 10 13:02:42.222698 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:42.222679 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f4c9f7664-s8rkz" Jun 10 13:02:42.233694 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:42.233674 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7f4c9f7664-s8rkz"] Jun 10 13:02:42.288531 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:42.288499 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jnnq\" (UniqueName: \"kubernetes.io/projected/fffe1a9b-aab8-485b-98cc-3b5870a78116-kube-api-access-9jnnq\") pod \"authorino-7f4c9f7664-s8rkz\" (UID: \"fffe1a9b-aab8-485b-98cc-3b5870a78116\") " pod="kuadrant-system/authorino-7f4c9f7664-s8rkz" Jun 10 13:02:42.389819 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:42.389792 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jnnq\" (UniqueName: \"kubernetes.io/projected/fffe1a9b-aab8-485b-98cc-3b5870a78116-kube-api-access-9jnnq\") pod \"authorino-7f4c9f7664-s8rkz\" (UID: \"fffe1a9b-aab8-485b-98cc-3b5870a78116\") " pod="kuadrant-system/authorino-7f4c9f7664-s8rkz" Jun 10 13:02:42.399816 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:42.399791 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jnnq\" (UniqueName: \"kubernetes.io/projected/fffe1a9b-aab8-485b-98cc-3b5870a78116-kube-api-access-9jnnq\") pod \"authorino-7f4c9f7664-s8rkz\" (UID: \"fffe1a9b-aab8-485b-98cc-3b5870a78116\") " pod="kuadrant-system/authorino-7f4c9f7664-s8rkz" Jun 10 13:02:42.532302 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:42.532270 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f4c9f7664-s8rkz" Jun 10 13:02:42.551560 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:42.551526 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7f4c9f7664-s8rkz"] Jun 10 13:02:42.664076 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:42.663914 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7f4c9f7664-s8rkz"] Jun 10 13:02:42.666727 ip-10-0-128-230 kubenswrapper[2581]: W0610 13:02:42.666697 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfffe1a9b_aab8_485b_98cc_3b5870a78116.slice/crio-d96181ceca6cd2c34735570b3add03de224bb0e1a4b39cbd07bae28423c72488 WatchSource:0}: Error finding container d96181ceca6cd2c34735570b3add03de224bb0e1a4b39cbd07bae28423c72488: Status 404 returned error can't find the container with id d96181ceca6cd2c34735570b3add03de224bb0e1a4b39cbd07bae28423c72488 Jun 10 13:02:43.019063 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:43.018975 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jbwpl" Jun 10 13:02:43.019063 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:43.018977 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f4c9f7664-s8rkz" event={"ID":"fffe1a9b-aab8-485b-98cc-3b5870a78116","Type":"ContainerStarted","Data":"d96181ceca6cd2c34735570b3add03de224bb0e1a4b39cbd07bae28423c72488"} Jun 10 13:02:43.026913 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:43.026894 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jbwpl" Jun 10 13:02:43.094874 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:43.094837 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxthd\" (UniqueName: \"kubernetes.io/projected/f54a2157-e72a-45ec-9463-75befe8fd328-kube-api-access-vxthd\") pod \"f54a2157-e72a-45ec-9463-75befe8fd328\" (UID: \"f54a2157-e72a-45ec-9463-75befe8fd328\") " Jun 10 13:02:43.096973 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:43.096952 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f54a2157-e72a-45ec-9463-75befe8fd328-kube-api-access-vxthd" (OuterVolumeSpecName: "kube-api-access-vxthd") pod "f54a2157-e72a-45ec-9463-75befe8fd328" (UID: "f54a2157-e72a-45ec-9463-75befe8fd328"). InnerVolumeSpecName "kube-api-access-vxthd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:02:43.196266 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:43.196239 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxthd\" (UniqueName: \"kubernetes.io/projected/f54a2157-e72a-45ec-9463-75befe8fd328-kube-api-access-vxthd\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:02:43.973382 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:43.973347 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-cgx62"] Jun 10 13:02:43.973549 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:43.973517 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-cgx62" podUID="25c43eaf-b062-484d-9991-a8d724c77902" containerName="authorino" containerID="cri-o://8b1a4ba3f6badda783a9747e0e98d54cb1404e69acde9de0fa90f9364c1c8533" gracePeriod=30 Jun 10 13:02:44.025450 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:44.025414 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f4c9f7664-s8rkz" event={"ID":"fffe1a9b-aab8-485b-98cc-3b5870a78116","Type":"ContainerStarted","Data":"fcfbb483b690e9011d5fdc057e12459c62c1dbebd272779118b0ee3751a1d442"} Jun 10 13:02:44.025852 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:44.025426 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jbwpl" Jun 10 13:02:44.025852 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:44.025483 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-7f4c9f7664-s8rkz" podUID="fffe1a9b-aab8-485b-98cc-3b5870a78116" containerName="authorino" containerID="cri-o://fcfbb483b690e9011d5fdc057e12459c62c1dbebd272779118b0ee3751a1d442" gracePeriod=30 Jun 10 13:02:44.044446 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:44.044389 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7f4c9f7664-s8rkz" podStartSLOduration=1.5738680299999999 podStartE2EDuration="2.044373405s" totalImagesPullingTime="470.505375ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:02:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:02:44.042902022 +0000 UTC m=+825.783116503" watchObservedRunningTime="2026-06-10 13:02:44.044373405 +0000 UTC m=+825.784587886" Jun 10 13:02:44.153211 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:44.153157 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jbwpl"] Jun 10 13:02:44.160517 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:44.160475 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jbwpl"] Jun 10 13:02:44.301001 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:44.300972 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-cgx62" Jun 10 13:02:44.304072 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:44.304054 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f4c9f7664-s8rkz" Jun 10 13:02:44.406489 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:44.406461 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngn9b\" (UniqueName: \"kubernetes.io/projected/25c43eaf-b062-484d-9991-a8d724c77902-kube-api-access-ngn9b\") pod \"25c43eaf-b062-484d-9991-a8d724c77902\" (UID: \"25c43eaf-b062-484d-9991-a8d724c77902\") " Jun 10 13:02:44.406489 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:44.406495 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jnnq\" (UniqueName: \"kubernetes.io/projected/fffe1a9b-aab8-485b-98cc-3b5870a78116-kube-api-access-9jnnq\") pod \"fffe1a9b-aab8-485b-98cc-3b5870a78116\" (UID: \"fffe1a9b-aab8-485b-98cc-3b5870a78116\") " Jun 10 13:02:44.408721 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:44.408681 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25c43eaf-b062-484d-9991-a8d724c77902-kube-api-access-ngn9b" (OuterVolumeSpecName: "kube-api-access-ngn9b") pod "25c43eaf-b062-484d-9991-a8d724c77902" (UID: "25c43eaf-b062-484d-9991-a8d724c77902"). InnerVolumeSpecName "kube-api-access-ngn9b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:02:44.408846 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:44.408759 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fffe1a9b-aab8-485b-98cc-3b5870a78116-kube-api-access-9jnnq" (OuterVolumeSpecName: "kube-api-access-9jnnq") pod "fffe1a9b-aab8-485b-98cc-3b5870a78116" (UID: "fffe1a9b-aab8-485b-98cc-3b5870a78116"). InnerVolumeSpecName "kube-api-access-9jnnq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:02:44.508014 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:44.507935 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngn9b\" (UniqueName: \"kubernetes.io/projected/25c43eaf-b062-484d-9991-a8d724c77902-kube-api-access-ngn9b\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:02:44.508014 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:44.507964 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jnnq\" (UniqueName: \"kubernetes.io/projected/fffe1a9b-aab8-485b-98cc-3b5870a78116-kube-api-access-9jnnq\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:02:44.718296 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:44.718260 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f54a2157-e72a-45ec-9463-75befe8fd328" path="/var/lib/kubelet/pods/f54a2157-e72a-45ec-9463-75befe8fd328/volumes" Jun 10 13:02:45.032132 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:45.032096 2581 generic.go:356] "Generic (PLEG): container finished" podID="25c43eaf-b062-484d-9991-a8d724c77902" containerID="8b1a4ba3f6badda783a9747e0e98d54cb1404e69acde9de0fa90f9364c1c8533" exitCode=0 Jun 10 13:02:45.032619 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:45.032139 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-cgx62" Jun 10 13:02:45.032619 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:45.032216 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-cgx62" event={"ID":"25c43eaf-b062-484d-9991-a8d724c77902","Type":"ContainerDied","Data":"8b1a4ba3f6badda783a9747e0e98d54cb1404e69acde9de0fa90f9364c1c8533"} Jun 10 13:02:45.032619 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:45.032254 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-cgx62" event={"ID":"25c43eaf-b062-484d-9991-a8d724c77902","Type":"ContainerDied","Data":"aecafb194aa53d4163bb74cdacaf93183126d72f7f44466de1e48fdd8e1b445c"} Jun 10 13:02:45.032619 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:45.032270 2581 scope.go:117] "RemoveContainer" containerID="8b1a4ba3f6badda783a9747e0e98d54cb1404e69acde9de0fa90f9364c1c8533" Jun 10 13:02:45.033653 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:45.033637 2581 generic.go:356] "Generic (PLEG): container finished" podID="fffe1a9b-aab8-485b-98cc-3b5870a78116" containerID="fcfbb483b690e9011d5fdc057e12459c62c1dbebd272779118b0ee3751a1d442" exitCode=0 Jun 10 13:02:45.033710 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:45.033661 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f4c9f7664-s8rkz" event={"ID":"fffe1a9b-aab8-485b-98cc-3b5870a78116","Type":"ContainerDied","Data":"fcfbb483b690e9011d5fdc057e12459c62c1dbebd272779118b0ee3751a1d442"} Jun 10 13:02:45.033710 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:45.033691 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7f4c9f7664-s8rkz" event={"ID":"fffe1a9b-aab8-485b-98cc-3b5870a78116","Type":"ContainerDied","Data":"d96181ceca6cd2c34735570b3add03de224bb0e1a4b39cbd07bae28423c72488"} Jun 10 13:02:45.033710 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:45.033693 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7f4c9f7664-s8rkz" Jun 10 13:02:45.041305 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:45.041291 2581 scope.go:117] "RemoveContainer" containerID="8b1a4ba3f6badda783a9747e0e98d54cb1404e69acde9de0fa90f9364c1c8533" Jun 10 13:02:45.041572 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:02:45.041554 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b1a4ba3f6badda783a9747e0e98d54cb1404e69acde9de0fa90f9364c1c8533\": container with ID starting with 8b1a4ba3f6badda783a9747e0e98d54cb1404e69acde9de0fa90f9364c1c8533 not found: ID does not exist" containerID="8b1a4ba3f6badda783a9747e0e98d54cb1404e69acde9de0fa90f9364c1c8533" Jun 10 13:02:45.041614 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:45.041580 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b1a4ba3f6badda783a9747e0e98d54cb1404e69acde9de0fa90f9364c1c8533"} err="failed to get container status \"8b1a4ba3f6badda783a9747e0e98d54cb1404e69acde9de0fa90f9364c1c8533\": rpc error: code = NotFound desc = could not find container \"8b1a4ba3f6badda783a9747e0e98d54cb1404e69acde9de0fa90f9364c1c8533\": container with ID starting with 8b1a4ba3f6badda783a9747e0e98d54cb1404e69acde9de0fa90f9364c1c8533 not found: ID does not exist" Jun 10 13:02:45.041614 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:45.041597 2581 scope.go:117] "RemoveContainer" containerID="fcfbb483b690e9011d5fdc057e12459c62c1dbebd272779118b0ee3751a1d442" Jun 10 13:02:45.049123 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:45.049105 2581 scope.go:117] "RemoveContainer" containerID="fcfbb483b690e9011d5fdc057e12459c62c1dbebd272779118b0ee3751a1d442" Jun 10 13:02:45.049490 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:02:45.049454 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcfbb483b690e9011d5fdc057e12459c62c1dbebd272779118b0ee3751a1d442\": container with ID starting with fcfbb483b690e9011d5fdc057e12459c62c1dbebd272779118b0ee3751a1d442 not found: ID does not exist" containerID="fcfbb483b690e9011d5fdc057e12459c62c1dbebd272779118b0ee3751a1d442" Jun 10 13:02:45.049587 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:45.049488 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcfbb483b690e9011d5fdc057e12459c62c1dbebd272779118b0ee3751a1d442"} err="failed to get container status \"fcfbb483b690e9011d5fdc057e12459c62c1dbebd272779118b0ee3751a1d442\": rpc error: code = NotFound desc = could not find container \"fcfbb483b690e9011d5fdc057e12459c62c1dbebd272779118b0ee3751a1d442\": container with ID starting with fcfbb483b690e9011d5fdc057e12459c62c1dbebd272779118b0ee3751a1d442 not found: ID does not exist" Jun 10 13:02:45.054398 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:45.052767 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-cgx62"] Jun 10 13:02:45.059659 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:45.059639 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-cgx62"] Jun 10 13:02:45.072891 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:45.072872 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7f4c9f7664-s8rkz"] Jun 10 13:02:45.079012 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:45.078989 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7f4c9f7664-s8rkz"] Jun 10 13:02:46.718397 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:46.718367 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25c43eaf-b062-484d-9991-a8d724c77902" path="/var/lib/kubelet/pods/25c43eaf-b062-484d-9991-a8d724c77902/volumes" Jun 10 13:02:46.718844 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:46.718680 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fffe1a9b-aab8-485b-98cc-3b5870a78116" path="/var/lib/kubelet/pods/fffe1a9b-aab8-485b-98cc-3b5870a78116/volumes" Jun 10 13:02:47.506375 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:47.506334 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:02:47.605449 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:47.605414 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:02:47.695396 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:47.695338 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:02:47.784232 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:47.784119 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:02:51.230246 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:51.230198 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:02:51.322712 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:02:51.322673 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:03:19.892413 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:19.892371 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:03:20.127181 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:20.127126 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:03:20.659605 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:20.659562 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:03:20.786509 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:20.786467 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:03:20.985193 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:20.985086 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-kvdwp"] Jun 10 13:03:20.985546 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:20.985490 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25c43eaf-b062-484d-9991-a8d724c77902" containerName="authorino" Jun 10 13:03:20.985546 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:20.985507 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="25c43eaf-b062-484d-9991-a8d724c77902" containerName="authorino" Jun 10 13:03:20.985546 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:20.985530 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fffe1a9b-aab8-485b-98cc-3b5870a78116" containerName="authorino" Jun 10 13:03:20.985546 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:20.985535 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fffe1a9b-aab8-485b-98cc-3b5870a78116" containerName="authorino" Jun 10 13:03:20.985680 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:20.985598 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fffe1a9b-aab8-485b-98cc-3b5870a78116" containerName="authorino" Jun 10 13:03:20.985680 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:20.985609 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="25c43eaf-b062-484d-9991-a8d724c77902" containerName="authorino" Jun 10 13:03:20.988618 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:20.988594 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-kvdwp" Jun 10 13:03:20.991311 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:20.991283 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-jtf8j\"" Jun 10 13:03:20.991473 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:20.991334 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 10 13:03:21.005286 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:21.005257 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-kvdwp"] Jun 10 13:03:21.142147 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:21.142110 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhtt7\" (UniqueName: \"kubernetes.io/projected/8307898a-5c9e-42e2-914c-3c41ed8436a5-kube-api-access-dhtt7\") pod \"payload-pre-processing-579785bc64-kvdwp\" (UID: \"8307898a-5c9e-42e2-914c-3c41ed8436a5\") " pod="openshift-ingress/payload-pre-processing-579785bc64-kvdwp" Jun 10 13:03:21.242738 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:21.242647 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhtt7\" (UniqueName: \"kubernetes.io/projected/8307898a-5c9e-42e2-914c-3c41ed8436a5-kube-api-access-dhtt7\") pod \"payload-pre-processing-579785bc64-kvdwp\" (UID: \"8307898a-5c9e-42e2-914c-3c41ed8436a5\") " pod="openshift-ingress/payload-pre-processing-579785bc64-kvdwp" Jun 10 13:03:21.253030 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:21.253001 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhtt7\" (UniqueName: \"kubernetes.io/projected/8307898a-5c9e-42e2-914c-3c41ed8436a5-kube-api-access-dhtt7\") pod \"payload-pre-processing-579785bc64-kvdwp\" (UID: \"8307898a-5c9e-42e2-914c-3c41ed8436a5\") " pod="openshift-ingress/payload-pre-processing-579785bc64-kvdwp" Jun 10 13:03:21.301258 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:21.301228 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-kvdwp" Jun 10 13:03:21.449410 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:21.449363 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-kvdwp"] Jun 10 13:03:21.457869 ip-10-0-128-230 kubenswrapper[2581]: W0610 13:03:21.457818 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8307898a_5c9e_42e2_914c_3c41ed8436a5.slice/crio-a31614cfe5cca5147c7fa7ccc6f3b149a77fcde4b9628e0cebfd027ac4887c69 WatchSource:0}: Error finding container a31614cfe5cca5147c7fa7ccc6f3b149a77fcde4b9628e0cebfd027ac4887c69: Status 404 returned error can't find the container with id a31614cfe5cca5147c7fa7ccc6f3b149a77fcde4b9628e0cebfd027ac4887c69 Jun 10 13:03:22.215099 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:22.215018 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-kvdwp" event={"ID":"8307898a-5c9e-42e2-914c-3c41ed8436a5","Type":"ContainerStarted","Data":"a31614cfe5cca5147c7fa7ccc6f3b149a77fcde4b9628e0cebfd027ac4887c69"} Jun 10 13:03:24.226680 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:24.226640 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-kvdwp" event={"ID":"8307898a-5c9e-42e2-914c-3c41ed8436a5","Type":"ContainerStarted","Data":"645cd0d3e080859e6eb2e7ee011d91da8153a936b6aad867f4cd73fd15762b1f"} Jun 10 13:03:24.227059 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:24.226755 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-579785bc64-kvdwp" Jun 10 13:03:24.247058 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:24.246994 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-579785bc64-kvdwp" podStartSLOduration=1.6245176209999999 podStartE2EDuration="4.246973222s" totalImagesPullingTime="2.622455601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:03:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:03:24.244293982 +0000 UTC m=+865.984508463" watchObservedRunningTime="2026-06-10 13:03:24.246973222 +0000 UTC m=+865.987187704" Jun 10 13:03:28.246848 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:28.246819 2581 generic.go:356] "Generic (PLEG): container finished" podID="d7a97a42-4ec9-4eea-a743-9baddfd1f118" containerID="cf5f9fc43c2b3c44268998b3a30648ed43e4c5475ab034b94768a81a5a9f9f5c" exitCode=0 Jun 10 13:03:28.247230 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:28.246888 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" event={"ID":"d7a97a42-4ec9-4eea-a743-9baddfd1f118","Type":"ContainerDied","Data":"cf5f9fc43c2b3c44268998b3a30648ed43e4c5475ab034b94768a81a5a9f9f5c"} Jun 10 13:03:28.247277 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:28.247261 2581 scope.go:117] "RemoveContainer" containerID="cf5f9fc43c2b3c44268998b3a30648ed43e4c5475ab034b94768a81a5a9f9f5c" Jun 10 13:03:28.248252 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:28.248222 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jun 10 13:03:28.248354 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:28.248308 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jun 10 13:03:28.248417 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:28.248355 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jun 10 13:03:29.256938 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:29.256890 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" event={"ID":"d7a97a42-4ec9-4eea-a743-9baddfd1f118","Type":"ContainerStarted","Data":"fde7450a1361a61d70db1f303564b319078285b169f4330116e586c2a7a38d66"} Jun 10 13:03:30.214713 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:30.214679 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:03:30.214890 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:30.214869 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-4wqlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jun 10 13:03:30.214950 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:30.214926 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" podUID="d7a97a42-4ec9-4eea-a743-9baddfd1f118" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jun 10 13:03:31.215181 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:31.215131 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-4wqlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jun 10 13:03:31.215544 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:31.215207 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" podUID="d7a97a42-4ec9-4eea-a743-9baddfd1f118" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jun 10 13:03:32.215703 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:32.215666 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-4wqlm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jun 10 13:03:32.216089 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:32.215740 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" podUID="d7a97a42-4ec9-4eea-a743-9baddfd1f118" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jun 10 13:03:33.218645 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:33.218613 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:03:33.219110 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:33.218996 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:03:33.219683 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:33.219666 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-4wqlm" Jun 10 13:03:35.233602 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:35.233571 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-579785bc64-kvdwp" Jun 10 13:03:58.739922 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:58.739846 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgg5q_8a91de05-d0b9-4192-9583-8de8cf91c175/ovn-acl-logging/0.log" Jun 10 13:03:58.743998 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:03:58.743978 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgg5q_8a91de05-d0b9-4192-9583-8de8cf91c175/ovn-acl-logging/0.log" Jun 10 13:04:18.365336 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:18.365303 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6555bf4b74-nf69s"] Jun 10 13:04:18.370628 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:18.370610 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6555bf4b74-nf69s" Jun 10 13:04:18.374762 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:18.374741 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 10 13:04:18.374878 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:18.374775 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 10 13:04:18.374878 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:18.374741 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-n5gh5\"" Jun 10 13:04:18.378641 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:18.378617 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6555bf4b74-nf69s"] Jun 10 13:04:18.458379 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:18.458348 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgp92\" (UniqueName: \"kubernetes.io/projected/ebbd13c1-1e6b-463d-9bdc-04ed570d46c7-kube-api-access-vgp92\") pod \"authorino-6555bf4b74-nf69s\" (UID: \"ebbd13c1-1e6b-463d-9bdc-04ed570d46c7\") " pod="kuadrant-system/authorino-6555bf4b74-nf69s" Jun 10 13:04:18.458379 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:18.458381 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/ebbd13c1-1e6b-463d-9bdc-04ed570d46c7-oidc-ca\") pod \"authorino-6555bf4b74-nf69s\" (UID: \"ebbd13c1-1e6b-463d-9bdc-04ed570d46c7\") " pod="kuadrant-system/authorino-6555bf4b74-nf69s" Jun 10 13:04:18.458576 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:18.458402 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ebbd13c1-1e6b-463d-9bdc-04ed570d46c7-tls-cert\") pod \"authorino-6555bf4b74-nf69s\" (UID: \"ebbd13c1-1e6b-463d-9bdc-04ed570d46c7\") " pod="kuadrant-system/authorino-6555bf4b74-nf69s" Jun 10 13:04:18.558950 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:18.558919 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgp92\" (UniqueName: \"kubernetes.io/projected/ebbd13c1-1e6b-463d-9bdc-04ed570d46c7-kube-api-access-vgp92\") pod \"authorino-6555bf4b74-nf69s\" (UID: \"ebbd13c1-1e6b-463d-9bdc-04ed570d46c7\") " pod="kuadrant-system/authorino-6555bf4b74-nf69s" Jun 10 13:04:18.558950 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:18.558955 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/ebbd13c1-1e6b-463d-9bdc-04ed570d46c7-oidc-ca\") pod \"authorino-6555bf4b74-nf69s\" (UID: \"ebbd13c1-1e6b-463d-9bdc-04ed570d46c7\") " pod="kuadrant-system/authorino-6555bf4b74-nf69s" Jun 10 13:04:18.559206 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:18.558979 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ebbd13c1-1e6b-463d-9bdc-04ed570d46c7-tls-cert\") pod \"authorino-6555bf4b74-nf69s\" (UID: \"ebbd13c1-1e6b-463d-9bdc-04ed570d46c7\") " pod="kuadrant-system/authorino-6555bf4b74-nf69s" Jun 10 13:04:18.559581 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:18.559561 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/ebbd13c1-1e6b-463d-9bdc-04ed570d46c7-oidc-ca\") pod \"authorino-6555bf4b74-nf69s\" (UID: \"ebbd13c1-1e6b-463d-9bdc-04ed570d46c7\") " pod="kuadrant-system/authorino-6555bf4b74-nf69s" Jun 10 13:04:18.561632 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:18.561609 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ebbd13c1-1e6b-463d-9bdc-04ed570d46c7-tls-cert\") pod \"authorino-6555bf4b74-nf69s\" (UID: \"ebbd13c1-1e6b-463d-9bdc-04ed570d46c7\") " pod="kuadrant-system/authorino-6555bf4b74-nf69s" Jun 10 13:04:18.569247 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:18.569218 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgp92\" (UniqueName: \"kubernetes.io/projected/ebbd13c1-1e6b-463d-9bdc-04ed570d46c7-kube-api-access-vgp92\") pod \"authorino-6555bf4b74-nf69s\" (UID: \"ebbd13c1-1e6b-463d-9bdc-04ed570d46c7\") " pod="kuadrant-system/authorino-6555bf4b74-nf69s" Jun 10 13:04:18.690926 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:18.690833 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6555bf4b74-nf69s" Jun 10 13:04:18.831554 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:18.831529 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6555bf4b74-nf69s"] Jun 10 13:04:18.833743 ip-10-0-128-230 kubenswrapper[2581]: W0610 13:04:18.833701 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podebbd13c1_1e6b_463d_9bdc_04ed570d46c7.slice/crio-fb8a4c1b00c45edacfcacb68b7c976ba49a00f6036f60513ca31cce805d1512b WatchSource:0}: Error finding container fb8a4c1b00c45edacfcacb68b7c976ba49a00f6036f60513ca31cce805d1512b: Status 404 returned error can't find the container with id fb8a4c1b00c45edacfcacb68b7c976ba49a00f6036f60513ca31cce805d1512b Jun 10 13:04:18.835223 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:18.835203 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 13:04:19.504414 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:19.504313 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6555bf4b74-nf69s" event={"ID":"ebbd13c1-1e6b-463d-9bdc-04ed570d46c7","Type":"ContainerStarted","Data":"3b0b08e54fec50ac6d7cf182bb457b7bf0d0ecc0e752035b191061aefdeadb10"} Jun 10 13:04:19.504414 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:19.504370 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6555bf4b74-nf69s" event={"ID":"ebbd13c1-1e6b-463d-9bdc-04ed570d46c7","Type":"ContainerStarted","Data":"fb8a4c1b00c45edacfcacb68b7c976ba49a00f6036f60513ca31cce805d1512b"} Jun 10 13:04:19.524817 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:19.524769 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6555bf4b74-nf69s" podStartSLOduration=1.208495673 podStartE2EDuration="1.524753903s" totalImagesPullingTime="316.25823ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:04:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:04:19.522944044 +0000 UTC m=+921.263158524" watchObservedRunningTime="2026-06-10 13:04:19.524753903 +0000 UTC m=+921.264968384" Jun 10 13:04:19.957896 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:19.957865 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:04:21.087920 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:21.087886 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:04:27.822030 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:27.821995 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:04:28.132681 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:28.132639 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:04:44.954157 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:44.954115 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:04:53.418419 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:04:53.418371 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:05:05.262193 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:05:05.262128 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:05:14.937272 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:05:14.937223 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:05:34.818949 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:05:34.818876 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:06:00.339357 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:06:00.339323 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:06:14.240446 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:06:14.240399 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:06:23.556240 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:06:23.556208 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:06:25.172516 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:06:25.172473 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:06:25.239434 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:06:25.239404 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:07:24.966684 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:24.966604 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-79bb87cc7f-mzhk4"] Jun 10 13:07:24.970127 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:24.970110 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79bb87cc7f-mzhk4" Jun 10 13:07:24.980531 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:24.980506 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79bb87cc7f-mzhk4"] Jun 10 13:07:25.039704 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:25.039670 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb4n8\" (UniqueName: \"kubernetes.io/projected/76592c72-f4ec-4c1b-b5f9-80d850158697-kube-api-access-fb4n8\") pod \"authorino-79bb87cc7f-mzhk4\" (UID: \"76592c72-f4ec-4c1b-b5f9-80d850158697\") " pod="kuadrant-system/authorino-79bb87cc7f-mzhk4" Jun 10 13:07:25.039873 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:25.039758 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/76592c72-f4ec-4c1b-b5f9-80d850158697-tls-cert\") pod \"authorino-79bb87cc7f-mzhk4\" (UID: \"76592c72-f4ec-4c1b-b5f9-80d850158697\") " pod="kuadrant-system/authorino-79bb87cc7f-mzhk4" Jun 10 13:07:25.039873 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:25.039785 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/76592c72-f4ec-4c1b-b5f9-80d850158697-oidc-ca\") pod \"authorino-79bb87cc7f-mzhk4\" (UID: \"76592c72-f4ec-4c1b-b5f9-80d850158697\") " pod="kuadrant-system/authorino-79bb87cc7f-mzhk4" Jun 10 13:07:25.140261 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:25.140219 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb4n8\" (UniqueName: \"kubernetes.io/projected/76592c72-f4ec-4c1b-b5f9-80d850158697-kube-api-access-fb4n8\") pod \"authorino-79bb87cc7f-mzhk4\" (UID: \"76592c72-f4ec-4c1b-b5f9-80d850158697\") " pod="kuadrant-system/authorino-79bb87cc7f-mzhk4" Jun 10 13:07:25.140447 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:25.140319 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/76592c72-f4ec-4c1b-b5f9-80d850158697-tls-cert\") pod \"authorino-79bb87cc7f-mzhk4\" (UID: \"76592c72-f4ec-4c1b-b5f9-80d850158697\") " pod="kuadrant-system/authorino-79bb87cc7f-mzhk4" Jun 10 13:07:25.140447 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:25.140348 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/76592c72-f4ec-4c1b-b5f9-80d850158697-oidc-ca\") pod \"authorino-79bb87cc7f-mzhk4\" (UID: \"76592c72-f4ec-4c1b-b5f9-80d850158697\") " pod="kuadrant-system/authorino-79bb87cc7f-mzhk4" Jun 10 13:07:25.140923 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:25.140906 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/76592c72-f4ec-4c1b-b5f9-80d850158697-oidc-ca\") pod \"authorino-79bb87cc7f-mzhk4\" (UID: \"76592c72-f4ec-4c1b-b5f9-80d850158697\") " pod="kuadrant-system/authorino-79bb87cc7f-mzhk4" Jun 10 13:07:25.142992 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:25.142973 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/76592c72-f4ec-4c1b-b5f9-80d850158697-tls-cert\") pod \"authorino-79bb87cc7f-mzhk4\" (UID: \"76592c72-f4ec-4c1b-b5f9-80d850158697\") " pod="kuadrant-system/authorino-79bb87cc7f-mzhk4" Jun 10 13:07:25.151756 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:25.151727 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb4n8\" (UniqueName: \"kubernetes.io/projected/76592c72-f4ec-4c1b-b5f9-80d850158697-kube-api-access-fb4n8\") pod \"authorino-79bb87cc7f-mzhk4\" (UID: \"76592c72-f4ec-4c1b-b5f9-80d850158697\") " pod="kuadrant-system/authorino-79bb87cc7f-mzhk4" Jun 10 13:07:25.279114 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:25.279037 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79bb87cc7f-mzhk4" Jun 10 13:07:25.280992 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:25.280954 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:07:25.382202 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:25.382139 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:07:25.431121 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:25.431083 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79bb87cc7f-mzhk4"] Jun 10 13:07:25.434128 ip-10-0-128-230 kubenswrapper[2581]: W0610 13:07:25.434105 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod76592c72_f4ec_4c1b_b5f9_80d850158697.slice/crio-3766f018f969f73c9755e0ca23a473cd4534e6ca88b734dc483eff22c4089a24 WatchSource:0}: Error finding container 3766f018f969f73c9755e0ca23a473cd4534e6ca88b734dc483eff22c4089a24: Status 404 returned error can't find the container with id 3766f018f969f73c9755e0ca23a473cd4534e6ca88b734dc483eff22c4089a24 Jun 10 13:07:26.386044 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:26.386009 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79bb87cc7f-mzhk4" event={"ID":"76592c72-f4ec-4c1b-b5f9-80d850158697","Type":"ContainerStarted","Data":"702990f7f0aae960186fd978bf522941e02b656786fd0de0c13c400f70c35b77"} Jun 10 13:07:26.386501 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:26.386056 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79bb87cc7f-mzhk4" event={"ID":"76592c72-f4ec-4c1b-b5f9-80d850158697","Type":"ContainerStarted","Data":"3766f018f969f73c9755e0ca23a473cd4534e6ca88b734dc483eff22c4089a24"} Jun 10 13:07:26.485265 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:26.485225 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:07:26.572708 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:26.572650 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-79bb87cc7f-mzhk4" podStartSLOduration=2.113536813 podStartE2EDuration="2.57263312s" totalImagesPullingTime="459.096307ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:07:26.539196273 +0000 UTC m=+1108.279410751" watchObservedRunningTime="2026-06-10 13:07:26.57263312 +0000 UTC m=+1108.312847601" Jun 10 13:07:26.573345 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:26.573322 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6555bf4b74-nf69s"] Jun 10 13:07:26.573524 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:26.573504 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-6555bf4b74-nf69s" podUID="ebbd13c1-1e6b-463d-9bdc-04ed570d46c7" containerName="authorino" containerID="cri-o://3b0b08e54fec50ac6d7cf182bb457b7bf0d0ecc0e752035b191061aefdeadb10" gracePeriod=30 Jun 10 13:07:26.616621 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:26.616584 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:07:26.833106 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:26.833080 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6555bf4b74-nf69s" Jun 10 13:07:26.855689 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:26.855664 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ebbd13c1-1e6b-463d-9bdc-04ed570d46c7-tls-cert\") pod \"ebbd13c1-1e6b-463d-9bdc-04ed570d46c7\" (UID: \"ebbd13c1-1e6b-463d-9bdc-04ed570d46c7\") " Jun 10 13:07:26.855821 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:26.855702 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/ebbd13c1-1e6b-463d-9bdc-04ed570d46c7-oidc-ca\") pod \"ebbd13c1-1e6b-463d-9bdc-04ed570d46c7\" (UID: \"ebbd13c1-1e6b-463d-9bdc-04ed570d46c7\") " Jun 10 13:07:26.855821 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:26.855793 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgp92\" (UniqueName: \"kubernetes.io/projected/ebbd13c1-1e6b-463d-9bdc-04ed570d46c7-kube-api-access-vgp92\") pod \"ebbd13c1-1e6b-463d-9bdc-04ed570d46c7\" (UID: \"ebbd13c1-1e6b-463d-9bdc-04ed570d46c7\") " Jun 10 13:07:26.857923 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:26.857899 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebbd13c1-1e6b-463d-9bdc-04ed570d46c7-kube-api-access-vgp92" (OuterVolumeSpecName: "kube-api-access-vgp92") pod "ebbd13c1-1e6b-463d-9bdc-04ed570d46c7" (UID: "ebbd13c1-1e6b-463d-9bdc-04ed570d46c7"). InnerVolumeSpecName "kube-api-access-vgp92". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:07:26.860736 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:26.860714 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebbd13c1-1e6b-463d-9bdc-04ed570d46c7-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "ebbd13c1-1e6b-463d-9bdc-04ed570d46c7" (UID: "ebbd13c1-1e6b-463d-9bdc-04ed570d46c7"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 13:07:26.866654 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:26.866628 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebbd13c1-1e6b-463d-9bdc-04ed570d46c7-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ebbd13c1-1e6b-463d-9bdc-04ed570d46c7" (UID: "ebbd13c1-1e6b-463d-9bdc-04ed570d46c7"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 10 13:07:26.957395 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:26.957296 2581 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/ebbd13c1-1e6b-463d-9bdc-04ed570d46c7-oidc-ca\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:07:26.957395 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:26.957330 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgp92\" (UniqueName: \"kubernetes.io/projected/ebbd13c1-1e6b-463d-9bdc-04ed570d46c7-kube-api-access-vgp92\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:07:26.957395 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:26.957340 2581 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ebbd13c1-1e6b-463d-9bdc-04ed570d46c7-tls-cert\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:07:27.393626 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:27.393587 2581 generic.go:356] "Generic (PLEG): container finished" podID="ebbd13c1-1e6b-463d-9bdc-04ed570d46c7" containerID="3b0b08e54fec50ac6d7cf182bb457b7bf0d0ecc0e752035b191061aefdeadb10" exitCode=0 Jun 10 13:07:27.394014 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:27.393636 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6555bf4b74-nf69s" Jun 10 13:07:27.394014 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:27.393667 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6555bf4b74-nf69s" event={"ID":"ebbd13c1-1e6b-463d-9bdc-04ed570d46c7","Type":"ContainerDied","Data":"3b0b08e54fec50ac6d7cf182bb457b7bf0d0ecc0e752035b191061aefdeadb10"} Jun 10 13:07:27.394014 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:27.393711 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6555bf4b74-nf69s" event={"ID":"ebbd13c1-1e6b-463d-9bdc-04ed570d46c7","Type":"ContainerDied","Data":"fb8a4c1b00c45edacfcacb68b7c976ba49a00f6036f60513ca31cce805d1512b"} Jun 10 13:07:27.394014 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:27.393731 2581 scope.go:117] "RemoveContainer" containerID="3b0b08e54fec50ac6d7cf182bb457b7bf0d0ecc0e752035b191061aefdeadb10" Jun 10 13:07:27.403515 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:27.403498 2581 scope.go:117] "RemoveContainer" containerID="3b0b08e54fec50ac6d7cf182bb457b7bf0d0ecc0e752035b191061aefdeadb10" Jun 10 13:07:27.403788 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:07:27.403766 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b0b08e54fec50ac6d7cf182bb457b7bf0d0ecc0e752035b191061aefdeadb10\": container with ID starting with 3b0b08e54fec50ac6d7cf182bb457b7bf0d0ecc0e752035b191061aefdeadb10 not found: ID does not exist" containerID="3b0b08e54fec50ac6d7cf182bb457b7bf0d0ecc0e752035b191061aefdeadb10" Jun 10 13:07:27.403869 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:27.403797 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b0b08e54fec50ac6d7cf182bb457b7bf0d0ecc0e752035b191061aefdeadb10"} err="failed to get container status \"3b0b08e54fec50ac6d7cf182bb457b7bf0d0ecc0e752035b191061aefdeadb10\": rpc error: code = NotFound desc = could not find container \"3b0b08e54fec50ac6d7cf182bb457b7bf0d0ecc0e752035b191061aefdeadb10\": container with ID starting with 3b0b08e54fec50ac6d7cf182bb457b7bf0d0ecc0e752035b191061aefdeadb10 not found: ID does not exist" Jun 10 13:07:27.432194 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:27.432131 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6555bf4b74-nf69s"] Jun 10 13:07:27.441032 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:27.441003 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6555bf4b74-nf69s"] Jun 10 13:07:28.719256 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:07:28.719226 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebbd13c1-1e6b-463d-9bdc-04ed570d46c7" path="/var/lib/kubelet/pods/ebbd13c1-1e6b-463d-9bdc-04ed570d46c7/volumes" Jun 10 13:08:18.646320 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:08:18.646240 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:08:58.770420 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:08:58.770393 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgg5q_8a91de05-d0b9-4192-9583-8de8cf91c175/ovn-acl-logging/0.log" Jun 10 13:08:58.775790 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:08:58.775766 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgg5q_8a91de05-d0b9-4192-9583-8de8cf91c175/ovn-acl-logging/0.log" Jun 10 13:09:03.843786 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:09:03.843747 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:09:05.296008 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:09:05.295970 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:09:47.326856 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:09:47.326768 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:09:48.607715 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:09:48.607670 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:09:55.834340 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:09:55.834300 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:10:02.575559 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:02.575521 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:10:02.634563 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:02.634527 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:10:21.202834 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:21.202800 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:10:34.753846 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:34.753811 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:10:34.845126 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:34.845082 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:10:57.540319 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.540286 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6fd57bf9f-fsz9b"] Jun 10 13:10:57.540901 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.540853 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebbd13c1-1e6b-463d-9bdc-04ed570d46c7" containerName="authorino" Jun 10 13:10:57.540901 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.540874 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebbd13c1-1e6b-463d-9bdc-04ed570d46c7" containerName="authorino" Jun 10 13:10:57.541039 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.540983 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebbd13c1-1e6b-463d-9bdc-04ed570d46c7" containerName="authorino" Jun 10 13:10:57.543989 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.543967 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.546838 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.546816 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 10 13:10:57.546961 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.546819 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 10 13:10:57.547883 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.547864 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 10 13:10:57.547987 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.547935 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 10 13:10:57.548051 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.547980 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 10 13:10:57.548051 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.548041 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 10 13:10:57.548158 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.548080 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-j2fn2\"" Jun 10 13:10:57.548241 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.548224 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 10 13:10:57.554403 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.554386 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 10 13:10:57.569473 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.569449 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fd57bf9f-fsz9b"] Jun 10 13:10:57.597615 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.597574 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:10:57.632299 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.632276 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-oauth-serving-cert\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.632393 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.632304 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-service-ca\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.632393 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.632323 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvn8n\" (UniqueName: \"kubernetes.io/projected/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-kube-api-access-nvn8n\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.632393 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.632346 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-trusted-ca-bundle\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.632499 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.632406 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-console-config\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.632499 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.632443 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-console-serving-cert\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.632565 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.632499 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-console-oauth-config\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.698454 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.698422 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:10:57.733536 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.733507 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-oauth-serving-cert\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.733638 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.733541 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-service-ca\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.733638 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.733559 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvn8n\" (UniqueName: \"kubernetes.io/projected/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-kube-api-access-nvn8n\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.733638 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.733586 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-trusted-ca-bundle\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.733638 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.733613 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-console-config\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.733842 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.733646 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-console-serving-cert\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.733842 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.733701 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-console-oauth-config\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.734403 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.734355 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-oauth-serving-cert\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.734506 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.734418 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-console-config\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.734506 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.734487 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-trusted-ca-bundle\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.734612 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.734418 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-service-ca\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.736818 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.736796 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-console-oauth-config\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.737020 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.737000 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-console-serving-cert\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.744009 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.743987 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvn8n\" (UniqueName: \"kubernetes.io/projected/a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8-kube-api-access-nvn8n\") pod \"console-6fd57bf9f-fsz9b\" (UID: \"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8\") " pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.854030 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.853936 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:10:57.997145 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:57.997111 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fd57bf9f-fsz9b"] Jun 10 13:10:58.409673 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:58.409634 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fd57bf9f-fsz9b" event={"ID":"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8","Type":"ContainerStarted","Data":"a4c0d85f269b01ab7d2de8e3daa2bf42e40058c07a542e2a1f5eb76254d7444b"} Jun 10 13:10:58.409860 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:58.409689 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fd57bf9f-fsz9b" event={"ID":"a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8","Type":"ContainerStarted","Data":"21d6c6af7924278b2c48041b05e81e913bfcdd537a5f00ddf3af3e122b0f9e3f"} Jun 10 13:10:58.434221 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:58.434157 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6fd57bf9f-fsz9b" podStartSLOduration=1.434142399 podStartE2EDuration="1.434142399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:10:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:10:58.431610244 +0000 UTC m=+1320.171824725" watchObservedRunningTime="2026-06-10 13:10:58.434142399 +0000 UTC m=+1320.174356934" Jun 10 13:10:58.711931 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:10:58.711835 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:05.433438 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:05.433405 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:05.495509 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:05.495473 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:05.587156 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:05.587116 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:05.681296 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:05.681264 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:05.805129 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:05.805051 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:05.895686 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:05.895650 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:07.855061 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:07.855008 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:11:07.855550 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:07.855077 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:11:07.860136 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:07.860109 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:11:08.467911 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:08.467886 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6fd57bf9f-fsz9b" Jun 10 13:11:08.719042 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:08.718948 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:08.815814 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:08.815782 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:08.941616 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:08.941583 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:09.040222 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:09.040104 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:22.303882 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:22.303794 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:22.427997 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:22.427968 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:22.528694 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:22.528658 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:29.317466 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:29.317422 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:29.379541 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:29.379506 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:37.459154 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:37.459122 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:37.563091 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:37.563052 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:37.666660 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:37.666619 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:37.768747 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:37.768660 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:54.231926 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:54.231893 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:54.351976 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:54.351929 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:11:54.457064 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:11:54.457026 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:12:01.325859 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:12:01.325816 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:12:01.388878 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:12:01.388849 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:12:23.163083 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:12:23.163047 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:12:23.852522 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:12:23.852475 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:12:23.947578 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:12:23.947547 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:12:24.090579 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:12:24.090541 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:12:30.998784 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:12:30.998739 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:12:31.099578 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:12:31.099536 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:12:31.200035 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:12:31.199999 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:12:31.374448 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:12:31.374412 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:12:38.570367 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:12:38.570327 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:12:38.629637 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:12:38.629591 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:13:58.799991 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:13:58.799880 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgg5q_8a91de05-d0b9-4192-9583-8de8cf91c175/ovn-acl-logging/0.log" Jun 10 13:13:58.806505 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:13:58.806487 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgg5q_8a91de05-d0b9-4192-9583-8de8cf91c175/ovn-acl-logging/0.log" Jun 10 13:15:00.173076 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:00.173036 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29684955-dt2hm"] Jun 10 13:15:00.176816 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:00.176798 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29684955-dt2hm" Jun 10 13:15:00.179679 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:00.179659 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-6lm29\"" Jun 10 13:15:00.203687 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:00.203658 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29684955-dt2hm"] Jun 10 13:15:00.298931 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:00.298897 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tktqd\" (UniqueName: \"kubernetes.io/projected/0206ed3c-a821-4ab4-ba50-d64d3450b922-kube-api-access-tktqd\") pod \"maas-api-key-cleanup-29684955-dt2hm\" (UID: \"0206ed3c-a821-4ab4-ba50-d64d3450b922\") " pod="opendatahub/maas-api-key-cleanup-29684955-dt2hm" Jun 10 13:15:00.399482 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:00.399447 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tktqd\" (UniqueName: \"kubernetes.io/projected/0206ed3c-a821-4ab4-ba50-d64d3450b922-kube-api-access-tktqd\") pod \"maas-api-key-cleanup-29684955-dt2hm\" (UID: \"0206ed3c-a821-4ab4-ba50-d64d3450b922\") " pod="opendatahub/maas-api-key-cleanup-29684955-dt2hm" Jun 10 13:15:00.410748 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:00.410712 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tktqd\" (UniqueName: \"kubernetes.io/projected/0206ed3c-a821-4ab4-ba50-d64d3450b922-kube-api-access-tktqd\") pod \"maas-api-key-cleanup-29684955-dt2hm\" (UID: \"0206ed3c-a821-4ab4-ba50-d64d3450b922\") " pod="opendatahub/maas-api-key-cleanup-29684955-dt2hm" Jun 10 13:15:00.487718 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:00.487634 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29684955-dt2hm" Jun 10 13:15:00.627407 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:00.627364 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29684955-dt2hm"] Jun 10 13:15:00.629760 ip-10-0-128-230 kubenswrapper[2581]: W0610 13:15:00.629734 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0206ed3c_a821_4ab4_ba50_d64d3450b922.slice/crio-3c45c3caed2ed2f4cd0ba4100a71dcaa375121ca92ac935a62e76c0400fa9844 WatchSource:0}: Error finding container 3c45c3caed2ed2f4cd0ba4100a71dcaa375121ca92ac935a62e76c0400fa9844: Status 404 returned error can't find the container with id 3c45c3caed2ed2f4cd0ba4100a71dcaa375121ca92ac935a62e76c0400fa9844 Jun 10 13:15:00.631647 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:00.631629 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 13:15:01.594475 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:01.594435 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29684955-dt2hm" event={"ID":"0206ed3c-a821-4ab4-ba50-d64d3450b922","Type":"ContainerStarted","Data":"3c45c3caed2ed2f4cd0ba4100a71dcaa375121ca92ac935a62e76c0400fa9844"} Jun 10 13:15:03.607303 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:03.607268 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29684955-dt2hm" event={"ID":"0206ed3c-a821-4ab4-ba50-d64d3450b922","Type":"ContainerStarted","Data":"e58f99bb5b921c0e9520bb28c58bb86dc65004ba99863465e85ce56bb52a404b"} Jun 10 13:15:03.625837 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:03.625790 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29684955-dt2hm" podStartSLOduration=1.014194123 podStartE2EDuration="3.625775333s" totalImagesPullingTime="2.61158121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:15:03.624970417 +0000 UTC m=+1565.365184913" watchObservedRunningTime="2026-06-10 13:15:03.625775333 +0000 UTC m=+1565.365989813" Jun 10 13:15:04.614018 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:04.613983 2581 generic.go:356] "Generic (PLEG): container finished" podID="0206ed3c-a821-4ab4-ba50-d64d3450b922" containerID="e58f99bb5b921c0e9520bb28c58bb86dc65004ba99863465e85ce56bb52a404b" exitCode=0 Jun 10 13:15:04.614438 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:04.614077 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29684955-dt2hm" event={"ID":"0206ed3c-a821-4ab4-ba50-d64d3450b922","Type":"ContainerDied","Data":"e58f99bb5b921c0e9520bb28c58bb86dc65004ba99863465e85ce56bb52a404b"} Jun 10 13:15:05.756591 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:05.756565 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29684955-dt2hm" Jun 10 13:15:05.850733 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:05.850690 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tktqd\" (UniqueName: \"kubernetes.io/projected/0206ed3c-a821-4ab4-ba50-d64d3450b922-kube-api-access-tktqd\") pod \"0206ed3c-a821-4ab4-ba50-d64d3450b922\" (UID: \"0206ed3c-a821-4ab4-ba50-d64d3450b922\") " Jun 10 13:15:05.853057 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:05.853033 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0206ed3c-a821-4ab4-ba50-d64d3450b922-kube-api-access-tktqd" (OuterVolumeSpecName: "kube-api-access-tktqd") pod "0206ed3c-a821-4ab4-ba50-d64d3450b922" (UID: "0206ed3c-a821-4ab4-ba50-d64d3450b922"). InnerVolumeSpecName "kube-api-access-tktqd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:15:05.952048 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:05.951970 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tktqd\" (UniqueName: \"kubernetes.io/projected/0206ed3c-a821-4ab4-ba50-d64d3450b922-kube-api-access-tktqd\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:15:06.626244 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:06.626213 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29684955-dt2hm" Jun 10 13:15:06.626244 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:06.626225 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29684955-dt2hm" event={"ID":"0206ed3c-a821-4ab4-ba50-d64d3450b922","Type":"ContainerDied","Data":"3c45c3caed2ed2f4cd0ba4100a71dcaa375121ca92ac935a62e76c0400fa9844"} Jun 10 13:15:06.626449 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:06.626278 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c45c3caed2ed2f4cd0ba4100a71dcaa375121ca92ac935a62e76c0400fa9844" Jun 10 13:15:40.275864 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:15:40.275817 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:17:13.191941 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:17:13.191904 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:17:57.604123 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:17:57.604089 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:17:57.664936 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:17:57.664901 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:18:58.830300 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:18:58.830185 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgg5q_8a91de05-d0b9-4192-9583-8de8cf91c175/ovn-acl-logging/0.log" Jun 10 13:18:58.837895 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:18:58.837873 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgg5q_8a91de05-d0b9-4192-9583-8de8cf91c175/ovn-acl-logging/0.log" Jun 10 13:19:41.852084 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:19:41.852050 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb"] Jun 10 13:19:41.852628 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:19:41.852270 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb" podUID="4bc87560-33a3-4edc-aa18-abe71940a3e5" containerName="manager" containerID="cri-o://17d4d042256994a905df9457904a97ec5a8933f0bc36b9b0c5ea416d1601c3b4" gracePeriod=10 Jun 10 13:19:41.980911 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:19:41.980882 2581 generic.go:356] "Generic (PLEG): container finished" podID="4bc87560-33a3-4edc-aa18-abe71940a3e5" containerID="17d4d042256994a905df9457904a97ec5a8933f0bc36b9b0c5ea416d1601c3b4" exitCode=0 Jun 10 13:19:41.981029 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:19:41.980967 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb" event={"ID":"4bc87560-33a3-4edc-aa18-abe71940a3e5","Type":"ContainerDied","Data":"17d4d042256994a905df9457904a97ec5a8933f0bc36b9b0c5ea416d1601c3b4"} Jun 10 13:19:42.108858 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:19:42.108836 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb" Jun 10 13:19:42.205759 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:19:42.205734 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4bc87560-33a3-4edc-aa18-abe71940a3e5-extensions-socket-volume\") pod \"4bc87560-33a3-4edc-aa18-abe71940a3e5\" (UID: \"4bc87560-33a3-4edc-aa18-abe71940a3e5\") " Jun 10 13:19:42.205927 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:19:42.205775 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvrhj\" (UniqueName: \"kubernetes.io/projected/4bc87560-33a3-4edc-aa18-abe71940a3e5-kube-api-access-nvrhj\") pod \"4bc87560-33a3-4edc-aa18-abe71940a3e5\" (UID: \"4bc87560-33a3-4edc-aa18-abe71940a3e5\") " Jun 10 13:19:42.206092 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:19:42.206070 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bc87560-33a3-4edc-aa18-abe71940a3e5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4bc87560-33a3-4edc-aa18-abe71940a3e5" (UID: "4bc87560-33a3-4edc-aa18-abe71940a3e5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 13:19:42.209014 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:19:42.208981 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bc87560-33a3-4edc-aa18-abe71940a3e5-kube-api-access-nvrhj" (OuterVolumeSpecName: "kube-api-access-nvrhj") pod "4bc87560-33a3-4edc-aa18-abe71940a3e5" (UID: "4bc87560-33a3-4edc-aa18-abe71940a3e5"). InnerVolumeSpecName "kube-api-access-nvrhj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:19:42.306561 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:19:42.306530 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4bc87560-33a3-4edc-aa18-abe71940a3e5-extensions-socket-volume\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:19:42.306561 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:19:42.306555 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvrhj\" (UniqueName: \"kubernetes.io/projected/4bc87560-33a3-4edc-aa18-abe71940a3e5-kube-api-access-nvrhj\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:19:42.987522 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:19:42.987430 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb" event={"ID":"4bc87560-33a3-4edc-aa18-abe71940a3e5","Type":"ContainerDied","Data":"6e75776c9eea635030c11e1ac7cbbd7519d465712d4c96bf9139293fe2af9471"} Jun 10 13:19:42.987522 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:19:42.987491 2581 scope.go:117] "RemoveContainer" containerID="17d4d042256994a905df9457904a97ec5a8933f0bc36b9b0c5ea416d1601c3b4" Jun 10 13:19:42.987522 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:19:42.987490 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb" Jun 10 13:19:43.009255 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:19:43.009229 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb"] Jun 10 13:19:43.014375 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:19:43.014355 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-knnrb"] Jun 10 13:19:44.718741 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:19:44.718707 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bc87560-33a3-4edc-aa18-abe71940a3e5" path="/var/lib/kubelet/pods/4bc87560-33a3-4edc-aa18-abe71940a3e5/volumes" Jun 10 13:20:47.933615 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:47.933583 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jgwbs"] Jun 10 13:20:47.934059 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:47.934020 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bc87560-33a3-4edc-aa18-abe71940a3e5" containerName="manager" Jun 10 13:20:47.934059 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:47.934032 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bc87560-33a3-4edc-aa18-abe71940a3e5" containerName="manager" Jun 10 13:20:47.934059 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:47.934042 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0206ed3c-a821-4ab4-ba50-d64d3450b922" containerName="cleanup" Jun 10 13:20:47.934059 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:47.934051 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0206ed3c-a821-4ab4-ba50-d64d3450b922" containerName="cleanup" Jun 10 13:20:47.934209 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:47.934108 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0206ed3c-a821-4ab4-ba50-d64d3450b922" containerName="cleanup" Jun 10 13:20:47.934209 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:47.934121 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bc87560-33a3-4edc-aa18-abe71940a3e5" containerName="manager" Jun 10 13:20:47.937006 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:47.936990 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jgwbs" Jun 10 13:20:47.939706 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:47.939689 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2ndqb\"" Jun 10 13:20:47.949498 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:47.949472 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jgwbs"] Jun 10 13:20:48.019108 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:48.019070 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8glqv\" (UniqueName: \"kubernetes.io/projected/e7c7a664-78ba-48d4-bfc7-c5335e69ad01-kube-api-access-8glqv\") pod \"kuadrant-operator-controller-manager-86878c769f-jgwbs\" (UID: \"e7c7a664-78ba-48d4-bfc7-c5335e69ad01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jgwbs" Jun 10 13:20:48.019311 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:48.019212 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7c7a664-78ba-48d4-bfc7-c5335e69ad01-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jgwbs\" (UID: \"e7c7a664-78ba-48d4-bfc7-c5335e69ad01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jgwbs" Jun 10 13:20:48.120333 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:48.120302 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8glqv\" (UniqueName: \"kubernetes.io/projected/e7c7a664-78ba-48d4-bfc7-c5335e69ad01-kube-api-access-8glqv\") pod \"kuadrant-operator-controller-manager-86878c769f-jgwbs\" (UID: \"e7c7a664-78ba-48d4-bfc7-c5335e69ad01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jgwbs" Jun 10 13:20:48.120495 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:48.120435 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7c7a664-78ba-48d4-bfc7-c5335e69ad01-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jgwbs\" (UID: \"e7c7a664-78ba-48d4-bfc7-c5335e69ad01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jgwbs" Jun 10 13:20:48.120793 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:48.120774 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7c7a664-78ba-48d4-bfc7-c5335e69ad01-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jgwbs\" (UID: \"e7c7a664-78ba-48d4-bfc7-c5335e69ad01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jgwbs" Jun 10 13:20:48.131422 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:48.131401 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8glqv\" (UniqueName: \"kubernetes.io/projected/e7c7a664-78ba-48d4-bfc7-c5335e69ad01-kube-api-access-8glqv\") pod \"kuadrant-operator-controller-manager-86878c769f-jgwbs\" (UID: \"e7c7a664-78ba-48d4-bfc7-c5335e69ad01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jgwbs" Jun 10 13:20:48.246790 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:48.246726 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jgwbs" Jun 10 13:20:48.394374 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:48.394313 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jgwbs"] Jun 10 13:20:48.397040 ip-10-0-128-230 kubenswrapper[2581]: W0610 13:20:48.397010 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7c7a664_78ba_48d4_bfc7_c5335e69ad01.slice/crio-7a82be3d1e038a88434b00257062ed980d1c2a3f0c14652daf2dc1d3954fe5d1 WatchSource:0}: Error finding container 7a82be3d1e038a88434b00257062ed980d1c2a3f0c14652daf2dc1d3954fe5d1: Status 404 returned error can't find the container with id 7a82be3d1e038a88434b00257062ed980d1c2a3f0c14652daf2dc1d3954fe5d1 Jun 10 13:20:49.305939 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:49.305905 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jgwbs" event={"ID":"e7c7a664-78ba-48d4-bfc7-c5335e69ad01","Type":"ContainerStarted","Data":"85c3ab91e958f5fb15c13440f18a0398e2d3ee01ad92bcdb7d3815cdf692cd4e"} Jun 10 13:20:49.306344 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:49.305957 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jgwbs" event={"ID":"e7c7a664-78ba-48d4-bfc7-c5335e69ad01","Type":"ContainerStarted","Data":"7a82be3d1e038a88434b00257062ed980d1c2a3f0c14652daf2dc1d3954fe5d1"} Jun 10 13:20:49.306344 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:49.306024 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jgwbs" Jun 10 13:20:49.331531 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:20:49.331477 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jgwbs" podStartSLOduration=2.331461468 podStartE2EDuration="2.331461468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:20:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:20:49.328722807 +0000 UTC m=+1911.068937291" watchObservedRunningTime="2026-06-10 13:20:49.331461468 +0000 UTC m=+1911.071675948" Jun 10 13:21:00.313066 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:21:00.313039 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jgwbs" Jun 10 13:21:58.492078 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:21:58.492025 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:23:00.649851 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:23:00.649810 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:23:58.862111 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:23:58.862083 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgg5q_8a91de05-d0b9-4192-9583-8de8cf91c175/ovn-acl-logging/0.log" Jun 10 13:23:58.869929 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:23:58.869909 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgg5q_8a91de05-d0b9-4192-9583-8de8cf91c175/ovn-acl-logging/0.log" Jun 10 13:25:29.303611 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:25:29.303578 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:25:29.368053 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:25:29.368011 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:28:58.896321 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:28:58.896200 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgg5q_8a91de05-d0b9-4192-9583-8de8cf91c175/ovn-acl-logging/0.log" Jun 10 13:28:58.906182 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:28:58.906147 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wgg5q_8a91de05-d0b9-4192-9583-8de8cf91c175/ovn-acl-logging/0.log" Jun 10 13:29:09.687975 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:29:09.687936 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:29:20.949374 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:29:20.949298 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:29:32.333785 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:29:32.333735 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:29:39.786455 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:29:39.786426 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:00.143728 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:00.143691 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29684970-29l75"] Jun 10 13:30:00.147527 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:00.147504 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29684970-29l75" Jun 10 13:30:00.150387 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:00.150366 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-6lm29\"" Jun 10 13:30:00.170801 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:00.170768 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29684970-29l75"] Jun 10 13:30:00.258262 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:00.258226 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrwbw\" (UniqueName: \"kubernetes.io/projected/fd34e541-7614-487a-bb1f-29b9049de585-kube-api-access-vrwbw\") pod \"maas-api-key-cleanup-29684970-29l75\" (UID: \"fd34e541-7614-487a-bb1f-29b9049de585\") " pod="opendatahub/maas-api-key-cleanup-29684970-29l75" Jun 10 13:30:00.359514 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:00.359470 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrwbw\" (UniqueName: \"kubernetes.io/projected/fd34e541-7614-487a-bb1f-29b9049de585-kube-api-access-vrwbw\") pod \"maas-api-key-cleanup-29684970-29l75\" (UID: \"fd34e541-7614-487a-bb1f-29b9049de585\") " pod="opendatahub/maas-api-key-cleanup-29684970-29l75" Jun 10 13:30:00.369943 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:00.369913 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrwbw\" (UniqueName: \"kubernetes.io/projected/fd34e541-7614-487a-bb1f-29b9049de585-kube-api-access-vrwbw\") pod \"maas-api-key-cleanup-29684970-29l75\" (UID: \"fd34e541-7614-487a-bb1f-29b9049de585\") " pod="opendatahub/maas-api-key-cleanup-29684970-29l75" Jun 10 13:30:00.457555 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:00.457450 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29684970-29l75" Jun 10 13:30:00.602408 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:00.602379 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29684970-29l75"] Jun 10 13:30:00.604989 ip-10-0-128-230 kubenswrapper[2581]: W0610 13:30:00.604962 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd34e541_7614_487a_bb1f_29b9049de585.slice/crio-bc15c787b7f40bcccd98de170ce0a291158c2f8b580032c1a288327570eb3da0 WatchSource:0}: Error finding container bc15c787b7f40bcccd98de170ce0a291158c2f8b580032c1a288327570eb3da0: Status 404 returned error can't find the container with id bc15c787b7f40bcccd98de170ce0a291158c2f8b580032c1a288327570eb3da0 Jun 10 13:30:01.052121 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:01.052076 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29684970-29l75" event={"ID":"fd34e541-7614-487a-bb1f-29b9049de585","Type":"ContainerStarted","Data":"b444ca95e30972a1bf7b8e2e151e5004cb6633ca62eedfe741001bcd03f09227"} Jun 10 13:30:01.052354 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:01.052143 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29684970-29l75" event={"ID":"fd34e541-7614-487a-bb1f-29b9049de585","Type":"ContainerStarted","Data":"bc15c787b7f40bcccd98de170ce0a291158c2f8b580032c1a288327570eb3da0"} Jun 10 13:30:01.069576 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:01.069516 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29684970-29l75" podStartSLOduration=1.069497093 podStartE2EDuration="1.069497093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:30:01.069151484 +0000 UTC m=+2462.809365964" watchObservedRunningTime="2026-06-10 13:30:01.069497093 +0000 UTC m=+2462.809711575" Jun 10 13:30:02.058454 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:02.058421 2581 generic.go:356] "Generic (PLEG): container finished" podID="fd34e541-7614-487a-bb1f-29b9049de585" containerID="b444ca95e30972a1bf7b8e2e151e5004cb6633ca62eedfe741001bcd03f09227" exitCode=0 Jun 10 13:30:02.058838 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:02.058472 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29684970-29l75" event={"ID":"fd34e541-7614-487a-bb1f-29b9049de585","Type":"ContainerDied","Data":"b444ca95e30972a1bf7b8e2e151e5004cb6633ca62eedfe741001bcd03f09227"} Jun 10 13:30:03.196865 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:03.196834 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29684970-29l75" Jun 10 13:30:03.286237 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:03.286203 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrwbw\" (UniqueName: \"kubernetes.io/projected/fd34e541-7614-487a-bb1f-29b9049de585-kube-api-access-vrwbw\") pod \"fd34e541-7614-487a-bb1f-29b9049de585\" (UID: \"fd34e541-7614-487a-bb1f-29b9049de585\") " Jun 10 13:30:03.288407 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:03.288383 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd34e541-7614-487a-bb1f-29b9049de585-kube-api-access-vrwbw" (OuterVolumeSpecName: "kube-api-access-vrwbw") pod "fd34e541-7614-487a-bb1f-29b9049de585" (UID: "fd34e541-7614-487a-bb1f-29b9049de585"). InnerVolumeSpecName "kube-api-access-vrwbw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:30:03.386863 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:03.386814 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrwbw\" (UniqueName: \"kubernetes.io/projected/fd34e541-7614-487a-bb1f-29b9049de585-kube-api-access-vrwbw\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:30:04.056109 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:04.056061 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:04.069705 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:04.069679 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29684970-29l75" event={"ID":"fd34e541-7614-487a-bb1f-29b9049de585","Type":"ContainerDied","Data":"bc15c787b7f40bcccd98de170ce0a291158c2f8b580032c1a288327570eb3da0"} Jun 10 13:30:04.069838 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:04.069723 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29684970-29l75" Jun 10 13:30:04.069838 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:04.069725 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc15c787b7f40bcccd98de170ce0a291158c2f8b580032c1a288327570eb3da0" Jun 10 13:30:13.746298 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:13.746259 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:13.847934 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:13.847887 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:17.131413 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:17.131375 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:18.731939 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.731904 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl"] Jun 10 13:30:18.732564 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.732396 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd34e541-7614-487a-bb1f-29b9049de585" containerName="cleanup" Jun 10 13:30:18.732564 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.732410 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd34e541-7614-487a-bb1f-29b9049de585" containerName="cleanup" Jun 10 13:30:18.732564 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.732485 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd34e541-7614-487a-bb1f-29b9049de585" containerName="cleanup" Jun 10 13:30:18.736087 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.736064 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.739486 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.739461 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-74870f6f-openshift-default-dockercfg-fphgx\"" Jun 10 13:30:18.749663 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.749638 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl"] Jun 10 13:30:18.780518 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.780484 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:18.827334 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.827306 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-workload-socket\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.827447 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.827335 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-token\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.827544 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.827524 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-workload-certs\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.827600 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.827583 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-podinfo\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.827655 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.827605 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istiod-ca-cert\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.827655 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.827626 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-credential-socket\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.827767 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.827654 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-data\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.827767 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.827749 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-envoy\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.827872 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.827784 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7zsr\" (UniqueName: \"kubernetes.io/projected/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-kube-api-access-v7zsr\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.885414 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.885381 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:18.928873 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.928840 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-workload-socket\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.928873 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.928874 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-token\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.929113 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.928916 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-workload-certs\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.929113 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.928939 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-podinfo\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.929113 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.928956 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istiod-ca-cert\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.929113 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.928974 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-credential-socket\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.929113 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.928992 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-data\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.929113 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.929020 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-envoy\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.929113 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.929054 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7zsr\" (UniqueName: \"kubernetes.io/projected/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-kube-api-access-v7zsr\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.929483 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.929330 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-workload-socket\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.929483 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.929371 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-workload-certs\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.929483 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.929459 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-credential-socket\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.929620 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.929599 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-data\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.929746 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.929729 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istiod-ca-cert\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.931519 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.931500 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-envoy\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.931717 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.931697 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-podinfo\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.948668 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.948639 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-token\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.953471 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.953451 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7zsr\" (UniqueName: \"kubernetes.io/projected/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-kube-api-access-v7zsr\") pod \"e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:18.980849 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:18.980823 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:19.049249 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:19.049150 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:19.197603 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:19.197577 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl"] Jun 10 13:30:19.199120 ip-10-0-128-230 kubenswrapper[2581]: W0610 13:30:19.199095 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda693eb5c_ccdd_4cb5_a60f_8e3c8dcd7acf.slice/crio-e5a117ad7ece0f70edcd23cd1c6c7329a2c1cd692fa9031f8e2c64ced28fe2a5 WatchSource:0}: Error finding container e5a117ad7ece0f70edcd23cd1c6c7329a2c1cd692fa9031f8e2c64ced28fe2a5: Status 404 returned error can't find the container with id e5a117ad7ece0f70edcd23cd1c6c7329a2c1cd692fa9031f8e2c64ced28fe2a5 Jun 10 13:30:19.201181 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:19.201136 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jun 10 13:30:19.201244 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:19.201225 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jun 10 13:30:19.201281 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:19.201259 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jun 10 13:30:20.158851 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:20.158815 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" event={"ID":"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf","Type":"ContainerStarted","Data":"82a1add04916a6bdbe61f99115e8c98ecb3b7ab2630d3d54db537180f68338b5"} Jun 10 13:30:20.159266 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:20.158868 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" event={"ID":"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf","Type":"ContainerStarted","Data":"e5a117ad7ece0f70edcd23cd1c6c7329a2c1cd692fa9031f8e2c64ced28fe2a5"} Jun 10 13:30:20.184101 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:20.184043 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" podStartSLOduration=2.184025909 podStartE2EDuration="2.184025909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:30:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:30:20.18125947 +0000 UTC m=+2481.921473954" watchObservedRunningTime="2026-06-10 13:30:20.184025909 +0000 UTC m=+2481.924240389" Jun 10 13:30:21.049505 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:21.049468 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:21.054536 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:21.054510 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:21.163613 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:21.163580 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:21.164590 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:21.164571 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:22.329486 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:22.329453 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:25.051333 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:25.051298 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:25.193454 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:25.193424 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl"] Jun 10 13:30:25.193668 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:25.193621 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" podUID="a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf" containerName="istio-proxy" containerID="cri-o://82a1add04916a6bdbe61f99115e8c98ecb3b7ab2630d3d54db537180f68338b5" gracePeriod=30 Jun 10 13:30:25.205471 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:25.205440 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:25.309339 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:25.309253 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:26.727404 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:26.727367 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:30.441696 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.441673 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:30.543524 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.543455 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-data\") pod \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " Jun 10 13:30:30.543524 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.543501 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-podinfo\") pod \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " Jun 10 13:30:30.543524 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.543523 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-workload-certs\") pod \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " Jun 10 13:30:30.543789 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.543557 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-token\") pod \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " Jun 10 13:30:30.543789 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.543584 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-workload-socket\") pod \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " Jun 10 13:30:30.543789 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.543623 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-envoy\") pod \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " Jun 10 13:30:30.543789 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.543649 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v7zsr\" (UniqueName: \"kubernetes.io/projected/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-kube-api-access-v7zsr\") pod \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " Jun 10 13:30:30.543789 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.543708 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-data" (OuterVolumeSpecName: "istio-data") pod "a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf" (UID: "a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 13:30:30.543789 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.543738 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-credential-socket\") pod \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " Jun 10 13:30:30.543789 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.543773 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istiod-ca-cert\") pod \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\" (UID: \"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf\") " Jun 10 13:30:30.544248 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.543923 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf" (UID: "a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 13:30:30.544248 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.543931 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf" (UID: "a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 13:30:30.544248 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.544080 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf" (UID: "a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 13:30:30.544248 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.544160 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-data\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:30:30.544248 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.544204 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-workload-certs\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:30:30.544248 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.544218 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-workload-socket\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:30:30.544720 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.544281 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf" (UID: "a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 13:30:30.545973 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.545946 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf" (UID: "a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 10 13:30:30.546113 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.546037 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-kube-api-access-v7zsr" (OuterVolumeSpecName: "kube-api-access-v7zsr") pod "a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf" (UID: "a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf"). InnerVolumeSpecName "kube-api-access-v7zsr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:30:30.546113 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.546075 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-token" (OuterVolumeSpecName: "istio-token") pod "a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf" (UID: "a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:30:30.546244 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.546097 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf" (UID: "a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 13:30:30.644826 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.644797 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-token\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:30:30.644826 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.644822 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-envoy\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:30:30.644826 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.644831 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7zsr\" (UniqueName: \"kubernetes.io/projected/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-kube-api-access-v7zsr\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:30:30.645043 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.644840 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-credential-socket\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:30:30.645043 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.644848 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istiod-ca-cert\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:30:30.645043 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:30.644858 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf-istio-podinfo\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:30:31.217951 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:31.217918 2581 generic.go:356] "Generic (PLEG): container finished" podID="a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf" containerID="82a1add04916a6bdbe61f99115e8c98ecb3b7ab2630d3d54db537180f68338b5" exitCode=0 Jun 10 13:30:31.218115 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:31.217974 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" Jun 10 13:30:31.218115 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:31.217999 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" event={"ID":"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf","Type":"ContainerDied","Data":"82a1add04916a6bdbe61f99115e8c98ecb3b7ab2630d3d54db537180f68338b5"} Jun 10 13:30:31.218115 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:31.218056 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl" event={"ID":"a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf","Type":"ContainerDied","Data":"e5a117ad7ece0f70edcd23cd1c6c7329a2c1cd692fa9031f8e2c64ced28fe2a5"} Jun 10 13:30:31.218115 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:31.218079 2581 scope.go:117] "RemoveContainer" containerID="82a1add04916a6bdbe61f99115e8c98ecb3b7ab2630d3d54db537180f68338b5" Jun 10 13:30:31.227544 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:31.227524 2581 scope.go:117] "RemoveContainer" containerID="82a1add04916a6bdbe61f99115e8c98ecb3b7ab2630d3d54db537180f68338b5" Jun 10 13:30:31.227817 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:30:31.227798 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82a1add04916a6bdbe61f99115e8c98ecb3b7ab2630d3d54db537180f68338b5\": container with ID starting with 82a1add04916a6bdbe61f99115e8c98ecb3b7ab2630d3d54db537180f68338b5 not found: ID does not exist" containerID="82a1add04916a6bdbe61f99115e8c98ecb3b7ab2630d3d54db537180f68338b5" Jun 10 13:30:31.227887 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:31.227826 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82a1add04916a6bdbe61f99115e8c98ecb3b7ab2630d3d54db537180f68338b5"} err="failed to get container status \"82a1add04916a6bdbe61f99115e8c98ecb3b7ab2630d3d54db537180f68338b5\": rpc error: code = NotFound desc = could not find container \"82a1add04916a6bdbe61f99115e8c98ecb3b7ab2630d3d54db537180f68338b5\": container with ID starting with 82a1add04916a6bdbe61f99115e8c98ecb3b7ab2630d3d54db537180f68338b5 not found: ID does not exist" Jun 10 13:30:31.240298 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:31.240276 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl"] Jun 10 13:30:31.247400 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:31.247373 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-74870f6f-openshift-default-bbc67677d-knhrl"] Jun 10 13:30:31.859615 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:31.859588 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:31.963003 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:31.962974 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:32.062843 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.062811 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6"] Jun 10 13:30:32.063261 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.063245 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf" containerName="istio-proxy" Jun 10 13:30:32.063345 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.063265 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf" containerName="istio-proxy" Jun 10 13:30:32.063411 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.063356 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf" containerName="istio-proxy" Jun 10 13:30:32.067320 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.067298 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.070469 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.070442 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-74b7a3f6-openshift-default-dockercfg-ndd97\"" Jun 10 13:30:32.090799 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.090772 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6"] Jun 10 13:30:32.109390 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.109365 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:32.157609 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.157547 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istiod-ca-cert\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.157609 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.157580 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-workload-socket\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.157754 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.157685 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-token\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.157754 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.157715 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smtsn\" (UniqueName: \"kubernetes.io/projected/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-kube-api-access-smtsn\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.157754 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.157743 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-workload-certs\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.157850 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.157806 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-data\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.157850 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.157829 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-envoy\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.157914 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.157851 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-podinfo\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.157914 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.157872 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-credential-socket\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.210778 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.210748 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:32.258581 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.258554 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-token\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.258759 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.258585 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smtsn\" (UniqueName: \"kubernetes.io/projected/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-kube-api-access-smtsn\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.258759 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.258609 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-workload-certs\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.258759 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.258655 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-data\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.258759 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.258676 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-envoy\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.258759 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.258694 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-podinfo\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.258759 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.258723 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-credential-socket\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.258759 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.258752 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istiod-ca-cert\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.259136 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.258787 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-workload-socket\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.259136 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.259069 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-workload-certs\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.259271 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.259128 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-data\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.259271 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.259157 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-credential-socket\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.259387 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.259291 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-workload-socket\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.259598 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.259576 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istiod-ca-cert\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.261329 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.261309 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-envoy\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.261429 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.261410 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-podinfo\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.273107 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.273079 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-token\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.273251 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.273153 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smtsn\" (UniqueName: \"kubernetes.io/projected/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-kube-api-access-smtsn\") pod \"e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.318513 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.318472 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:32.379632 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.379593 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:32.416750 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.416658 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:32.538621 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.538595 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6"] Jun 10 13:30:32.540739 ip-10-0-128-230 kubenswrapper[2581]: W0610 13:30:32.540708 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod149aecbf_bea7_4d66_8ade_a99f8c9bbbc9.slice/crio-d13f9c74939ac2285df329db3504d72ae3a300b2086e53b79065ba6a2c33c4e4 WatchSource:0}: Error finding container d13f9c74939ac2285df329db3504d72ae3a300b2086e53b79065ba6a2c33c4e4: Status 404 returned error can't find the container with id d13f9c74939ac2285df329db3504d72ae3a300b2086e53b79065ba6a2c33c4e4 Jun 10 13:30:32.542918 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.542887 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jun 10 13:30:32.543004 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.542947 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jun 10 13:30:32.543004 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.542975 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jun 10 13:30:32.720490 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:32.720407 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf" path="/var/lib/kubelet/pods/a693eb5c-ccdd-4cb5-a60f-8e3c8dcd7acf/volumes" Jun 10 13:30:33.229983 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:33.229946 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" event={"ID":"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9","Type":"ContainerStarted","Data":"3cb2ff672251848661f1560b7972600b691a04154ff9e6128105a7d3888f9b17"} Jun 10 13:30:33.230497 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:33.230043 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" event={"ID":"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9","Type":"ContainerStarted","Data":"d13f9c74939ac2285df329db3504d72ae3a300b2086e53b79065ba6a2c33c4e4"} Jun 10 13:30:33.259374 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:33.259316 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" podStartSLOduration=1.259298962 podStartE2EDuration="1.259298962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:30:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:30:33.257425303 +0000 UTC m=+2494.997639798" watchObservedRunningTime="2026-06-10 13:30:33.259298962 +0000 UTC m=+2494.999513443" Jun 10 13:30:33.380035 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:33.380002 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:33.385638 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:33.385613 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:34.235230 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:34.235196 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:34.236146 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:34.236128 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:35.535371 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:35.535337 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:38.407386 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:38.407344 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:39.518102 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:39.518064 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6"] Jun 10 13:30:39.518605 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:39.518251 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" podUID="149aecbf-bea7-4d66-8ade-a99f8c9bbbc9" containerName="istio-proxy" containerID="cri-o://3cb2ff672251848661f1560b7972600b691a04154ff9e6128105a7d3888f9b17" gracePeriod=30 Jun 10 13:30:39.545672 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:39.545635 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:39.652441 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:39.652410 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:40.864359 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:40.864300 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:44.778520 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.778458 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:44.878134 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.878103 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-data\") pod \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " Jun 10 13:30:44.878134 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.878138 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-envoy\") pod \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " Jun 10 13:30:44.878378 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.878161 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-workload-socket\") pod \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " Jun 10 13:30:44.878378 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.878226 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-token\") pod \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " Jun 10 13:30:44.878378 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.878242 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smtsn\" (UniqueName: \"kubernetes.io/projected/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-kube-api-access-smtsn\") pod \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " Jun 10 13:30:44.878378 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.878278 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-workload-certs\") pod \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " Jun 10 13:30:44.878378 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.878320 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istiod-ca-cert\") pod \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " Jun 10 13:30:44.878378 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.878345 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-credential-socket\") pod \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " Jun 10 13:30:44.878752 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.878416 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-podinfo\") pod \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\" (UID: \"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9\") " Jun 10 13:30:44.878752 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.878435 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-data" (OuterVolumeSpecName: "istio-data") pod "149aecbf-bea7-4d66-8ade-a99f8c9bbbc9" (UID: "149aecbf-bea7-4d66-8ade-a99f8c9bbbc9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 13:30:44.878752 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.878552 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "149aecbf-bea7-4d66-8ade-a99f8c9bbbc9" (UID: "149aecbf-bea7-4d66-8ade-a99f8c9bbbc9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 13:30:44.878752 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.878700 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-data\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:30:44.878752 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.878719 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-workload-socket\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:30:44.878752 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.878736 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "149aecbf-bea7-4d66-8ade-a99f8c9bbbc9" (UID: "149aecbf-bea7-4d66-8ade-a99f8c9bbbc9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 13:30:44.879231 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.878758 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "149aecbf-bea7-4d66-8ade-a99f8c9bbbc9" (UID: "149aecbf-bea7-4d66-8ade-a99f8c9bbbc9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 10 13:30:44.879231 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.878817 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "149aecbf-bea7-4d66-8ade-a99f8c9bbbc9" (UID: "149aecbf-bea7-4d66-8ade-a99f8c9bbbc9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 13:30:44.880530 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.880501 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-token" (OuterVolumeSpecName: "istio-token") pod "149aecbf-bea7-4d66-8ade-a99f8c9bbbc9" (UID: "149aecbf-bea7-4d66-8ade-a99f8c9bbbc9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:30:44.880647 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.880588 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "149aecbf-bea7-4d66-8ade-a99f8c9bbbc9" (UID: "149aecbf-bea7-4d66-8ade-a99f8c9bbbc9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 10 13:30:44.880694 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.880661 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "149aecbf-bea7-4d66-8ade-a99f8c9bbbc9" (UID: "149aecbf-bea7-4d66-8ade-a99f8c9bbbc9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 10 13:30:44.880694 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.880677 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-kube-api-access-smtsn" (OuterVolumeSpecName: "kube-api-access-smtsn") pod "149aecbf-bea7-4d66-8ade-a99f8c9bbbc9" (UID: "149aecbf-bea7-4d66-8ade-a99f8c9bbbc9"). InnerVolumeSpecName "kube-api-access-smtsn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 10 13:30:44.979296 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.979252 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-envoy\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:30:44.979296 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.979288 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-token\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:30:44.979296 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.979301 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smtsn\" (UniqueName: \"kubernetes.io/projected/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-kube-api-access-smtsn\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:30:44.979554 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.979313 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-workload-certs\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:30:44.979554 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.979326 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istiod-ca-cert\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:30:44.979554 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.979337 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-credential-socket\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:30:44.979554 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:44.979347 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9-istio-podinfo\") on node \"ip-10-0-128-230.ec2.internal\" DevicePath \"\"" Jun 10 13:30:45.289750 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:45.289713 2581 generic.go:356] "Generic (PLEG): container finished" podID="149aecbf-bea7-4d66-8ade-a99f8c9bbbc9" containerID="3cb2ff672251848661f1560b7972600b691a04154ff9e6128105a7d3888f9b17" exitCode=0 Jun 10 13:30:45.289912 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:45.289756 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" Jun 10 13:30:45.289912 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:45.289801 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" event={"ID":"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9","Type":"ContainerDied","Data":"3cb2ff672251848661f1560b7972600b691a04154ff9e6128105a7d3888f9b17"} Jun 10 13:30:45.289912 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:45.289853 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6" event={"ID":"149aecbf-bea7-4d66-8ade-a99f8c9bbbc9","Type":"ContainerDied","Data":"d13f9c74939ac2285df329db3504d72ae3a300b2086e53b79065ba6a2c33c4e4"} Jun 10 13:30:45.289912 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:45.289874 2581 scope.go:117] "RemoveContainer" containerID="3cb2ff672251848661f1560b7972600b691a04154ff9e6128105a7d3888f9b17" Jun 10 13:30:45.299433 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:45.299416 2581 scope.go:117] "RemoveContainer" containerID="3cb2ff672251848661f1560b7972600b691a04154ff9e6128105a7d3888f9b17" Jun 10 13:30:45.299713 ip-10-0-128-230 kubenswrapper[2581]: E0610 13:30:45.299689 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cb2ff672251848661f1560b7972600b691a04154ff9e6128105a7d3888f9b17\": container with ID starting with 3cb2ff672251848661f1560b7972600b691a04154ff9e6128105a7d3888f9b17 not found: ID does not exist" containerID="3cb2ff672251848661f1560b7972600b691a04154ff9e6128105a7d3888f9b17" Jun 10 13:30:45.299810 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:45.299720 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cb2ff672251848661f1560b7972600b691a04154ff9e6128105a7d3888f9b17"} err="failed to get container status \"3cb2ff672251848661f1560b7972600b691a04154ff9e6128105a7d3888f9b17\": rpc error: code = NotFound desc = could not find container \"3cb2ff672251848661f1560b7972600b691a04154ff9e6128105a7d3888f9b17\": container with ID starting with 3cb2ff672251848661f1560b7972600b691a04154ff9e6128105a7d3888f9b17 not found: ID does not exist" Jun 10 13:30:45.324731 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:45.324709 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6"] Jun 10 13:30:45.328096 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:45.328077 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-74b7a3f6-openshift-default-b5ff5c474-5prb6"] Jun 10 13:30:46.699417 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:46.699328 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:46.718418 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:46.718390 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="149aecbf-bea7-4d66-8ade-a99f8c9bbbc9" path="/var/lib/kubelet/pods/149aecbf-bea7-4d66-8ade-a99f8c9bbbc9/volumes" Jun 10 13:30:46.757937 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:46.757907 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:30:52.972812 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:52.972779 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-79bb87cc7f-mzhk4_76592c72-f4ec-4c1b-b5f9-80d850158697/authorino/0.log" Jun 10 13:30:57.581041 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:57.580995 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-dd7b5d84c-fkx7x_97f8a76c-d790-4eae-a95d-c4c8e370ad0c/nginx/0.log" Jun 10 13:30:58.205559 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:58.205522 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29684955-dt2hm_0206ed3c-a821-4ab4-ba50-d64d3450b922/cleanup/0.log" Jun 10 13:30:58.311964 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:58.311935 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29684970-29l75_fd34e541-7614-487a-bb1f-29b9049de585/cleanup/0.log" Jun 10 13:30:58.655479 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:58.655443 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-hbpq7_6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a/manager/0.log" Jun 10 13:30:59.020521 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:59.020447 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bb4db9f45-zttsr_ba89288e-b39e-46ba-afc0-e5960356305a/copy-manifests/0.log" Jun 10 13:30:59.029110 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:30:59.029083 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bb4db9f45-zttsr_ba89288e-b39e-46ba-afc0-e5960356305a/manager/0.log" Jun 10 13:31:00.519104 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:00.519072 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-79bb87cc7f-mzhk4_76592c72-f4ec-4c1b-b5f9-80d850158697/authorino/0.log" Jun 10 13:31:01.005850 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:01.005822 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-gwwvm_553e33c9-29b7-4aff-8887-4f69997c0a84/registry-server/0.log" Jun 10 13:31:01.125472 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:01.125439 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-jgwbs_e7c7a664-78ba-48d4-bfc7-c5335e69ad01/manager/0.log" Jun 10 13:31:01.381142 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:01.381115 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vk784_7c46c69f-3d3c-408f-a7b6-ba558717e019/manager/0.log" Jun 10 13:31:01.720300 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:01.720224 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bck6g5m_2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d/istio-proxy/0.log" Jun 10 13:31:02.185569 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:02.185544 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-4wqlm_d7a97a42-4ec9-4eea-a743-9baddfd1f118/istio-proxy/1.log" Jun 10 13:31:02.318346 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:02.318321 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-kvdwp_8307898a-5c9e-42e2-914c-3c41ed8436a5/payload-pre-processing/0.log" Jun 10 13:31:02.558576 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:02.558504 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6ccc644f47-p5gx4_59eb3b55-c728-4a20-bc60-758e5ffc296f/router/0.log" Jun 10 13:31:07.953231 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:07.953200 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:31:08.134967 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:08.134930 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rjqbr/must-gather-7kzg9"] Jun 10 13:31:08.135408 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:08.135393 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="149aecbf-bea7-4d66-8ade-a99f8c9bbbc9" containerName="istio-proxy" Jun 10 13:31:08.135460 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:08.135409 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="149aecbf-bea7-4d66-8ade-a99f8c9bbbc9" containerName="istio-proxy" Jun 10 13:31:08.135496 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:08.135471 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="149aecbf-bea7-4d66-8ade-a99f8c9bbbc9" containerName="istio-proxy" Jun 10 13:31:08.138529 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:08.138513 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rjqbr/must-gather-7kzg9" Jun 10 13:31:08.141849 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:08.141828 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rjqbr\"/\"default-dockercfg-gcscx\"" Jun 10 13:31:08.142085 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:08.142071 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rjqbr\"/\"kube-root-ca.crt\"" Jun 10 13:31:08.144777 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:08.144755 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rjqbr\"/\"openshift-service-ca.crt\"" Jun 10 13:31:08.145054 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:08.145028 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:31:08.151950 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:08.151926 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rjqbr/must-gather-7kzg9"] Jun 10 13:31:08.182083 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:08.182036 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9521e930-a4b0-474a-9ce6-cd73cbc7fdaf-must-gather-output\") pod \"must-gather-7kzg9\" (UID: \"9521e930-a4b0-474a-9ce6-cd73cbc7fdaf\") " pod="openshift-must-gather-rjqbr/must-gather-7kzg9" Jun 10 13:31:08.182259 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:08.182204 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn2j2\" (UniqueName: \"kubernetes.io/projected/9521e930-a4b0-474a-9ce6-cd73cbc7fdaf-kube-api-access-pn2j2\") pod \"must-gather-7kzg9\" (UID: \"9521e930-a4b0-474a-9ce6-cd73cbc7fdaf\") " pod="openshift-must-gather-rjqbr/must-gather-7kzg9" Jun 10 13:31:08.239982 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:08.239874 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" limit=1024 Jun 10 13:31:08.282865 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:08.282836 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn2j2\" (UniqueName: \"kubernetes.io/projected/9521e930-a4b0-474a-9ce6-cd73cbc7fdaf-kube-api-access-pn2j2\") pod \"must-gather-7kzg9\" (UID: \"9521e930-a4b0-474a-9ce6-cd73cbc7fdaf\") " pod="openshift-must-gather-rjqbr/must-gather-7kzg9" Jun 10 13:31:08.282993 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:08.282912 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9521e930-a4b0-474a-9ce6-cd73cbc7fdaf-must-gather-output\") pod \"must-gather-7kzg9\" (UID: \"9521e930-a4b0-474a-9ce6-cd73cbc7fdaf\") " pod="openshift-must-gather-rjqbr/must-gather-7kzg9" Jun 10 13:31:08.283239 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:08.283221 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9521e930-a4b0-474a-9ce6-cd73cbc7fdaf-must-gather-output\") pod \"must-gather-7kzg9\" (UID: \"9521e930-a4b0-474a-9ce6-cd73cbc7fdaf\") " pod="openshift-must-gather-rjqbr/must-gather-7kzg9" Jun 10 13:31:08.292290 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:08.292262 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn2j2\" (UniqueName: \"kubernetes.io/projected/9521e930-a4b0-474a-9ce6-cd73cbc7fdaf-kube-api-access-pn2j2\") pod \"must-gather-7kzg9\" (UID: \"9521e930-a4b0-474a-9ce6-cd73cbc7fdaf\") " pod="openshift-must-gather-rjqbr/must-gather-7kzg9" Jun 10 13:31:08.447568 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:08.447537 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rjqbr/must-gather-7kzg9" Jun 10 13:31:08.588597 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:08.588571 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rjqbr/must-gather-7kzg9"] Jun 10 13:31:08.590653 ip-10-0-128-230 kubenswrapper[2581]: W0610 13:31:08.590625 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9521e930_a4b0_474a_9ce6_cd73cbc7fdaf.slice/crio-837a45fb99971d5ea9b4ffe7a8e3ab71526e400d47bb96ac03815ba0a8974b29 WatchSource:0}: Error finding container 837a45fb99971d5ea9b4ffe7a8e3ab71526e400d47bb96ac03815ba0a8974b29: Status 404 returned error can't find the container with id 837a45fb99971d5ea9b4ffe7a8e3ab71526e400d47bb96ac03815ba0a8974b29 Jun 10 13:31:08.592608 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:08.592591 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 10 13:31:09.411705 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:09.411654 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rjqbr/must-gather-7kzg9" event={"ID":"9521e930-a4b0-474a-9ce6-cd73cbc7fdaf","Type":"ContainerStarted","Data":"837a45fb99971d5ea9b4ffe7a8e3ab71526e400d47bb96ac03815ba0a8974b29"} Jun 10 13:31:10.420913 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:10.420879 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rjqbr/must-gather-7kzg9" event={"ID":"9521e930-a4b0-474a-9ce6-cd73cbc7fdaf","Type":"ContainerStarted","Data":"cd7ab2f2632a5cff51022329b26967735138e180456884be58ae9a8470733069"} Jun 10 13:31:10.421326 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:10.420927 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rjqbr/must-gather-7kzg9" event={"ID":"9521e930-a4b0-474a-9ce6-cd73cbc7fdaf","Type":"ContainerStarted","Data":"36973fe3918fd079d7c0d7294797e595aa482e86a82763aa6b1b1ed48b6d8231"} Jun 10 13:31:10.443523 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:10.443433 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rjqbr/must-gather-7kzg9" podStartSLOduration=1.670234107 podStartE2EDuration="2.443419266s" totalImagesPullingTime="773.185159ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-10 13:31:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:31:10.440541668 +0000 UTC m=+2532.180756149" watchObservedRunningTime="2026-06-10 13:31:10.443419266 +0000 UTC m=+2532.183633748" Jun 10 13:31:11.419592 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:11.419563 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h67vv_907bdd6d-de14-412d-b061-0d62232f8f70/global-pull-secret-syncer/0.log" Jun 10 13:31:11.520927 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:11.520893 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g2s2c_707bfefc-7461-4cf2-8847-cacc35dcadd4/konnectivity-agent/0.log" Jun 10 13:31:11.620024 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:11.619989 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-230.ec2.internal_d5cbefa49f4f0656a69349468959f722/haproxy/0.log" Jun 10 13:31:17.071129 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:17.071093 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-79bb87cc7f-mzhk4_76592c72-f4ec-4c1b-b5f9-80d850158697/authorino/0.log" Jun 10 13:31:17.316253 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:17.316111 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-gwwvm_553e33c9-29b7-4aff-8887-4f69997c0a84/registry-server/0.log" Jun 10 13:31:17.397811 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:17.397782 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-jgwbs_e7c7a664-78ba-48d4-bfc7-c5335e69ad01/manager/0.log" Jun 10 13:31:17.532700 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:17.532663 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vk784_7c46c69f-3d3c-408f-a7b6-ba558717e019/manager/0.log" Jun 10 13:31:19.239215 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:19.239157 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_62c9fa3e-6551-4381-a49e-01a640e70389/alertmanager/0.log" Jun 10 13:31:19.280868 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:19.280834 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_62c9fa3e-6551-4381-a49e-01a640e70389/config-reloader/0.log" Jun 10 13:31:19.329829 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:19.329795 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_62c9fa3e-6551-4381-a49e-01a640e70389/kube-rbac-proxy-web/0.log" Jun 10 13:31:19.367674 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:19.367641 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_62c9fa3e-6551-4381-a49e-01a640e70389/kube-rbac-proxy/0.log" Jun 10 13:31:19.397614 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:19.397500 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_62c9fa3e-6551-4381-a49e-01a640e70389/kube-rbac-proxy-metric/0.log" Jun 10 13:31:19.433675 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:19.433556 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_62c9fa3e-6551-4381-a49e-01a640e70389/prom-label-proxy/0.log" Jun 10 13:31:19.488312 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:19.488273 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_62c9fa3e-6551-4381-a49e-01a640e70389/init-config-reloader/0.log" Jun 10 13:31:19.568712 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:19.568665 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-74bbf69bbb-j24l2_f57139cf-a599-4dcd-9f59-263cff6bbec2/cluster-monitoring-operator/0.log" Jun 10 13:31:19.993083 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:19.993053 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nqh9k_2bdc3c6b-9206-495d-a0a8-142384bfa823/node-exporter/0.log" Jun 10 13:31:20.024696 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:20.024644 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nqh9k_2bdc3c6b-9206-495d-a0a8-142384bfa823/kube-rbac-proxy/0.log" Jun 10 13:31:20.062885 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:20.062840 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nqh9k_2bdc3c6b-9206-495d-a0a8-142384bfa823/init-textfile/0.log" Jun 10 13:31:20.111567 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:20.111540 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-krz75_93cd4396-77ee-40b7-be1f-79e58bb2fae5/kube-rbac-proxy-main/0.log" Jun 10 13:31:20.151793 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:20.151767 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-krz75_93cd4396-77ee-40b7-be1f-79e58bb2fae5/kube-rbac-proxy-self/0.log" Jun 10 13:31:20.190238 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:20.190187 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-krz75_93cd4396-77ee-40b7-be1f-79e58bb2fae5/openshift-state-metrics/0.log" Jun 10 13:31:20.259500 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:20.259416 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f9dfbb49-dd88-409a-b7a9-34f2fec68073/prometheus/0.log" Jun 10 13:31:20.292375 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:20.292347 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f9dfbb49-dd88-409a-b7a9-34f2fec68073/config-reloader/0.log" Jun 10 13:31:20.321553 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:20.321514 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f9dfbb49-dd88-409a-b7a9-34f2fec68073/thanos-sidecar/0.log" Jun 10 13:31:20.353000 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:20.352975 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f9dfbb49-dd88-409a-b7a9-34f2fec68073/kube-rbac-proxy-web/0.log" Jun 10 13:31:20.400518 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:20.400488 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f9dfbb49-dd88-409a-b7a9-34f2fec68073/kube-rbac-proxy/0.log" Jun 10 13:31:20.464946 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:20.464913 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f9dfbb49-dd88-409a-b7a9-34f2fec68073/kube-rbac-proxy-thanos/0.log" Jun 10 13:31:20.515343 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:20.515252 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f9dfbb49-dd88-409a-b7a9-34f2fec68073/init-config-reloader/0.log" Jun 10 13:31:20.572044 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:20.572009 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-s7lkj_d1cad173-0816-45cf-a41f-91efbe0fc87f/prometheus-operator/0.log" Jun 10 13:31:20.622423 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:20.622396 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-s7lkj_d1cad173-0816-45cf-a41f-91efbe0fc87f/kube-rbac-proxy/0.log" Jun 10 13:31:22.213555 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.213521 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d5d5bb78b-4lhwz_49614bc3-1721-4b77-90b6-8fcfa6db5ae1/networking-console-plugin/0.log" Jun 10 13:31:22.491267 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.491153 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5"] Jun 10 13:31:22.496591 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.496562 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" Jun 10 13:31:22.504804 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.504773 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5"] Jun 10 13:31:22.662595 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.662557 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a0d9a07-9139-4582-a213-60d15ec6c200-sys\") pod \"perf-node-gather-daemonset-m89l5\" (UID: \"0a0d9a07-9139-4582-a213-60d15ec6c200\") " pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" Jun 10 13:31:22.662860 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.662840 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z5nt\" (UniqueName: \"kubernetes.io/projected/0a0d9a07-9139-4582-a213-60d15ec6c200-kube-api-access-7z5nt\") pod \"perf-node-gather-daemonset-m89l5\" (UID: \"0a0d9a07-9139-4582-a213-60d15ec6c200\") " pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" Jun 10 13:31:22.662984 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.662969 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0a0d9a07-9139-4582-a213-60d15ec6c200-proc\") pod \"perf-node-gather-daemonset-m89l5\" (UID: \"0a0d9a07-9139-4582-a213-60d15ec6c200\") " pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" Jun 10 13:31:22.663101 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.663087 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0a0d9a07-9139-4582-a213-60d15ec6c200-lib-modules\") pod \"perf-node-gather-daemonset-m89l5\" (UID: \"0a0d9a07-9139-4582-a213-60d15ec6c200\") " pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" Jun 10 13:31:22.663238 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.663225 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0a0d9a07-9139-4582-a213-60d15ec6c200-podres\") pod \"perf-node-gather-daemonset-m89l5\" (UID: \"0a0d9a07-9139-4582-a213-60d15ec6c200\") " pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" Jun 10 13:31:22.764773 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.764678 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a0d9a07-9139-4582-a213-60d15ec6c200-sys\") pod \"perf-node-gather-daemonset-m89l5\" (UID: \"0a0d9a07-9139-4582-a213-60d15ec6c200\") " pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" Jun 10 13:31:22.765077 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.765056 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z5nt\" (UniqueName: \"kubernetes.io/projected/0a0d9a07-9139-4582-a213-60d15ec6c200-kube-api-access-7z5nt\") pod \"perf-node-gather-daemonset-m89l5\" (UID: \"0a0d9a07-9139-4582-a213-60d15ec6c200\") " pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" Jun 10 13:31:22.765252 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.765233 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0a0d9a07-9139-4582-a213-60d15ec6c200-proc\") pod \"perf-node-gather-daemonset-m89l5\" (UID: \"0a0d9a07-9139-4582-a213-60d15ec6c200\") " pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" Jun 10 13:31:22.765480 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.765461 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0a0d9a07-9139-4582-a213-60d15ec6c200-lib-modules\") pod \"perf-node-gather-daemonset-m89l5\" (UID: \"0a0d9a07-9139-4582-a213-60d15ec6c200\") " pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" Jun 10 13:31:22.765747 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.765621 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0a0d9a07-9139-4582-a213-60d15ec6c200-lib-modules\") pod \"perf-node-gather-daemonset-m89l5\" (UID: \"0a0d9a07-9139-4582-a213-60d15ec6c200\") " pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" Jun 10 13:31:22.765747 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.765497 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0a0d9a07-9139-4582-a213-60d15ec6c200-proc\") pod \"perf-node-gather-daemonset-m89l5\" (UID: \"0a0d9a07-9139-4582-a213-60d15ec6c200\") " pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" Jun 10 13:31:22.765747 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.765625 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0a0d9a07-9139-4582-a213-60d15ec6c200-podres\") pod \"perf-node-gather-daemonset-m89l5\" (UID: \"0a0d9a07-9139-4582-a213-60d15ec6c200\") " pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" Jun 10 13:31:22.765747 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.765728 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0a0d9a07-9139-4582-a213-60d15ec6c200-podres\") pod \"perf-node-gather-daemonset-m89l5\" (UID: \"0a0d9a07-9139-4582-a213-60d15ec6c200\") " pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" Jun 10 13:31:22.766020 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.765067 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a0d9a07-9139-4582-a213-60d15ec6c200-sys\") pod \"perf-node-gather-daemonset-m89l5\" (UID: \"0a0d9a07-9139-4582-a213-60d15ec6c200\") " pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" Jun 10 13:31:22.784190 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.775302 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z5nt\" (UniqueName: \"kubernetes.io/projected/0a0d9a07-9139-4582-a213-60d15ec6c200-kube-api-access-7z5nt\") pod \"perf-node-gather-daemonset-m89l5\" (UID: \"0a0d9a07-9139-4582-a213-60d15ec6c200\") " pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" Jun 10 13:31:22.814899 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.814861 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" Jun 10 13:31:22.995569 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:22.995354 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5"] Jun 10 13:31:22.998359 ip-10-0-128-230 kubenswrapper[2581]: W0610 13:31:22.998313 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0a0d9a07_9139_4582_a213_60d15ec6c200.slice/crio-cf68b84c3b1cd3124a7b0bc40cf008e4835e0ff059fb6ed648eac754726a2c7e WatchSource:0}: Error finding container cf68b84c3b1cd3124a7b0bc40cf008e4835e0ff059fb6ed648eac754726a2c7e: Status 404 returned error can't find the container with id cf68b84c3b1cd3124a7b0bc40cf008e4835e0ff059fb6ed648eac754726a2c7e Jun 10 13:31:23.519494 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:23.519454 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" event={"ID":"0a0d9a07-9139-4582-a213-60d15ec6c200","Type":"ContainerStarted","Data":"c8d05ac881ee43b15f4819436c0903960f4c62b8027a327b8bffb4a20ba85dea"} Jun 10 13:31:23.519860 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:23.519523 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" event={"ID":"0a0d9a07-9139-4582-a213-60d15ec6c200","Type":"ContainerStarted","Data":"cf68b84c3b1cd3124a7b0bc40cf008e4835e0ff059fb6ed648eac754726a2c7e"} Jun 10 13:31:23.519860 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:23.519574 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" Jun 10 13:31:23.538969 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:23.538912 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" podStartSLOduration=1.5388896239999998 podStartE2EDuration="1.538889624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-10 13:31:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-10 13:31:23.537984744 +0000 UTC m=+2545.278199224" watchObservedRunningTime="2026-06-10 13:31:23.538889624 +0000 UTC m=+2545.279104105" Jun 10 13:31:23.540506 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:23.540484 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6fd57bf9f-fsz9b_a5c88c0b-2ab2-43b5-87c3-1bbdadfb84f8/console/0.log" Jun 10 13:31:24.292064 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:24.292029 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78dd696465-vkhjm_debfe673-395b-4ec5-aa38-bcff94c48874/volume-data-source-validator/0.log" Jun 10 13:31:25.463468 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:25.463430 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2pwn5_d0c4887d-c2bc-4f1b-8c0c-29674956c41d/dns/0.log" Jun 10 13:31:25.494880 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:25.494852 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2pwn5_d0c4887d-c2bc-4f1b-8c0c-29674956c41d/kube-rbac-proxy/0.log" Jun 10 13:31:25.714270 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:25.714191 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mtkd7_462abeb8-aa7a-475d-9600-1f8f9d579c84/dns-node-resolver/0.log" Jun 10 13:31:26.424607 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:26.424570 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jmkgf_de769e15-3af9-4466-8023-7e09fa20df3e/node-ca/0.log" Jun 10 13:31:27.506476 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:27.506445 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bck6g5m_2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d/istio-proxy/0.log" Jun 10 13:31:27.762959 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:27.762870 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-4wqlm_d7a97a42-4ec9-4eea-a743-9baddfd1f118/istio-proxy/0.log" Jun 10 13:31:27.797918 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:27.797886 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-4wqlm_d7a97a42-4ec9-4eea-a743-9baddfd1f118/istio-proxy/1.log" Jun 10 13:31:27.846806 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:27.846780 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-kvdwp_8307898a-5c9e-42e2-914c-3c41ed8436a5/payload-pre-processing/0.log" Jun 10 13:31:27.935857 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:27.935828 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6ccc644f47-p5gx4_59eb3b55-c728-4a20-bc60-758e5ffc296f/router/0.log" Jun 10 13:31:28.630136 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:28.630107 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bxb7s_c4e62b9a-b59d-4df1-a3fb-8185eb2a5472/serve-healthcheck-canary/0.log" Jun 10 13:31:29.515515 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:29.515479 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pbn64_c50085d0-35f4-40c0-8048-69b018c82b63/kube-rbac-proxy/0.log" Jun 10 13:31:29.536131 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:29.536106 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rjqbr/perf-node-gather-daemonset-m89l5" Jun 10 13:31:29.546618 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:29.546591 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pbn64_c50085d0-35f4-40c0-8048-69b018c82b63/exporter/0.log" Jun 10 13:31:29.581737 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:29.581709 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pbn64_c50085d0-35f4-40c0-8048-69b018c82b63/extractor/0.log" Jun 10 13:31:32.226481 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:32.226442 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-dd7b5d84c-fkx7x_97f8a76c-d790-4eae-a95d-c4c8e370ad0c/nginx/0.log" Jun 10 13:31:32.462458 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:32.462429 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29684955-dt2hm_0206ed3c-a821-4ab4-ba50-d64d3450b922/cleanup/0.log" Jun 10 13:31:32.499585 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:32.499551 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29684970-29l75_fd34e541-7614-487a-bb1f-29b9049de585/cleanup/0.log" Jun 10 13:31:32.712207 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:32.712180 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-hbpq7_6c5e63c4-4fec-4cee-8b53-6be5cecb9d4a/manager/0.log" Jun 10 13:31:32.969977 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:32.969947 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bb4db9f45-zttsr_ba89288e-b39e-46ba-afc0-e5960356305a/manager/0.log" Jun 10 13:31:32.996834 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:32.996795 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bb4db9f45-zttsr_ba89288e-b39e-46ba-afc0-e5960356305a/copy-manifests/0.log" Jun 10 13:31:34.743582 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:34.743551 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7b975d4df8-6fjzg_5b84ae64-e59e-4712-8f23-1ec53d548e21/manager/0.log" Jun 10 13:31:36.131464 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:36.131426 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bck6g5m_2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d/istio-proxy/0.log" Jun 10 13:31:36.257241 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:36.257209 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-4wqlm_d7a97a42-4ec9-4eea-a743-9baddfd1f118/istio-proxy/0.log" Jun 10 13:31:36.298336 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:36.298300 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-4wqlm_d7a97a42-4ec9-4eea-a743-9baddfd1f118/istio-proxy/1.log" Jun 10 13:31:36.325708 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:36.325677 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-kvdwp_8307898a-5c9e-42e2-914c-3c41ed8436a5/payload-pre-processing/0.log" Jun 10 13:31:36.394001 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:36.393929 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6ccc644f47-p5gx4_59eb3b55-c728-4a20-bc60-758e5ffc296f/router/0.log" Jun 10 13:31:37.493661 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:37.493621 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bck6g5m_2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d/istio-proxy/0.log" Jun 10 13:31:37.626884 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:37.626823 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-4wqlm_d7a97a42-4ec9-4eea-a743-9baddfd1f118/istio-proxy/0.log" Jun 10 13:31:37.670607 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:37.670570 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-4wqlm_d7a97a42-4ec9-4eea-a743-9baddfd1f118/istio-proxy/1.log" Jun 10 13:31:37.700999 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:37.700968 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-kvdwp_8307898a-5c9e-42e2-914c-3c41ed8436a5/payload-pre-processing/0.log" Jun 10 13:31:37.772327 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:37.772260 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6ccc644f47-p5gx4_59eb3b55-c728-4a20-bc60-758e5ffc296f/router/0.log" Jun 10 13:31:39.523036 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:39.522957 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bck6g5m_2fee2f7e-d91a-4f9b-8848-7f9c347bcf0d/istio-proxy/0.log" Jun 10 13:31:39.662906 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:39.662877 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-4wqlm_d7a97a42-4ec9-4eea-a743-9baddfd1f118/istio-proxy/0.log" Jun 10 13:31:39.714762 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:39.714577 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-4wqlm_d7a97a42-4ec9-4eea-a743-9baddfd1f118/istio-proxy/1.log" Jun 10 13:31:39.740588 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:39.740551 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-kvdwp_8307898a-5c9e-42e2-914c-3c41ed8436a5/payload-pre-processing/0.log" Jun 10 13:31:39.819341 ip-10-0-128-230 kubenswrapper[2581]: I0610 13:31:39.819235 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6ccc644f47-p5gx4_59eb3b55-c728-4a20-bc60-758e5ffc296f/router/0.log"