Jun 15 03:06:15.332141 ip-10-0-143-185 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 15 03:06:15.332154 ip-10-0-143-185 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 15 03:06:15.332163 ip-10-0-143-185 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 15 03:06:15.332515 ip-10-0-143-185 systemd[1]: Failed to start Kubernetes Kubelet. Jun 15 03:06:25.581962 ip-10-0-143-185 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 15 03:06:25.581983 ip-10-0-143-185 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 4bc0d905d84d446d96873d3e91ddb922 -- Jun 15 03:08:51.596103 ip-10-0-143-185 systemd[1]: Starting Kubernetes Kubelet... Jun 15 03:08:51.894359 ip-10-0-143-185 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 15 03:08:51.894359 ip-10-0-143-185 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 15 03:08:51.894359 ip-10-0-143-185 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 15 03:08:51.894359 ip-10-0-143-185 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 15 03:08:51.894359 ip-10-0-143-185 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 15 03:08:51.895488 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.894987 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 15 03:08:51.899418 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899401 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 15 03:08:51.899418 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899417 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 03:08:51.899483 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899421 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 03:08:51.899483 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899425 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 03:08:51.899483 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899429 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 03:08:51.899483 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899433 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 03:08:51.899483 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899436 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 03:08:51.899483 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899444 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 03:08:51.899483 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899447 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 03:08:51.899483 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899451 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 03:08:51.899483 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899454 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 03:08:51.899483 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899457 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 03:08:51.899483 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899460 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 03:08:51.899483 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899463 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 03:08:51.899483 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899466 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 03:08:51.899483 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899468 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 03:08:51.899483 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899471 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 03:08:51.899483 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899473 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 03:08:51.899483 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899476 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 03:08:51.899483 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899479 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 03:08:51.899483 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899482 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 03:08:51.899954 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899484 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 03:08:51.899954 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899487 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 03:08:51.899954 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899490 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 03:08:51.899954 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899493 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 03:08:51.899954 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899496 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 03:08:51.899954 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899499 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 03:08:51.899954 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899501 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 03:08:51.899954 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899506 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 03:08:51.899954 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899511 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 03:08:51.899954 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899514 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 03:08:51.899954 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899516 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 03:08:51.899954 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899519 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 03:08:51.899954 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899522 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 03:08:51.899954 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899524 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 03:08:51.899954 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899527 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 03:08:51.899954 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899530 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 03:08:51.899954 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899532 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 03:08:51.899954 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899536 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 03:08:51.899954 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899538 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 03:08:51.899954 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899541 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 03:08:51.900439 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899543 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 03:08:51.900439 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899545 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 03:08:51.900439 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899548 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 03:08:51.900439 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899550 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 03:08:51.900439 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899554 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 03:08:51.900439 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899558 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 03:08:51.900439 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899561 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 03:08:51.900439 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899564 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 03:08:51.900439 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899566 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 03:08:51.900439 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899569 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 03:08:51.900439 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899572 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 03:08:51.900439 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899575 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 03:08:51.900439 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899577 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 03:08:51.900439 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899581 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 03:08:51.900439 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899584 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 03:08:51.900439 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899587 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 03:08:51.900439 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899589 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 03:08:51.900439 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899592 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 03:08:51.900439 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899594 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 03:08:51.900439 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899597 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 03:08:51.900945 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899601 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 03:08:51.900945 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899603 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 03:08:51.900945 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899606 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 03:08:51.900945 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899608 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 03:08:51.900945 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899611 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 03:08:51.900945 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899613 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 03:08:51.900945 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899616 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 03:08:51.900945 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899618 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 03:08:51.900945 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899621 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 03:08:51.900945 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899623 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 03:08:51.900945 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899626 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 03:08:51.900945 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899629 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 03:08:51.900945 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899632 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 03:08:51.900945 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899634 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 03:08:51.900945 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899637 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 03:08:51.900945 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899639 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 03:08:51.900945 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899642 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 03:08:51.900945 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899644 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 03:08:51.900945 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899647 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 03:08:51.900945 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899650 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 03:08:51.901425 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899652 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 03:08:51.901425 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899654 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 03:08:51.901425 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899657 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 03:08:51.901425 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899659 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 03:08:51.901425 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899662 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 03:08:51.901425 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899665 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 03:08:51.901425 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899668 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 03:08:51.901425 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899670 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 03:08:51.901425 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899673 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 03:08:51.901425 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899676 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 03:08:51.901425 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899678 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 03:08:51.901425 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899681 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 03:08:51.901425 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899684 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 03:08:51.901425 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899686 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 03:08:51.901425 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899689 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 03:08:51.901425 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899692 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 03:08:51.901425 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899696 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 03:08:51.901425 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.899698 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 03:08:51.901425 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900471 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 03:08:51.901903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900480 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 03:08:51.901903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900484 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 03:08:51.901903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900487 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 03:08:51.901903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900490 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 03:08:51.901903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900493 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 03:08:51.901903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900496 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 03:08:51.901903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900499 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 03:08:51.901903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900502 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 03:08:51.901903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900505 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 03:08:51.901903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900507 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 03:08:51.901903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900510 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 03:08:51.901903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900512 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 03:08:51.901903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900516 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 03:08:51.901903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900519 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 03:08:51.901903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900522 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 03:08:51.901903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900525 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 03:08:51.901903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900527 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 03:08:51.901903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900530 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 03:08:51.901903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900533 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 03:08:51.901903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900536 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 03:08:51.902407 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900538 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 03:08:51.902407 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900541 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 03:08:51.902407 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900544 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 03:08:51.902407 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900548 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 03:08:51.902407 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900551 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 03:08:51.902407 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900554 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 03:08:51.902407 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900557 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 03:08:51.902407 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900560 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 03:08:51.902407 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900562 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 03:08:51.902407 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900565 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 03:08:51.902407 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900567 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 03:08:51.902407 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900570 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 03:08:51.902407 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900572 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 03:08:51.902407 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900575 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 03:08:51.902407 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900577 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 03:08:51.902407 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900580 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 03:08:51.902407 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900583 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 03:08:51.902407 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900586 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 03:08:51.902407 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900588 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 03:08:51.902898 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900591 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 03:08:51.902898 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900593 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 03:08:51.902898 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900596 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 03:08:51.902898 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900598 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 03:08:51.902898 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900601 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 03:08:51.902898 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900604 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 03:08:51.902898 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900607 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 03:08:51.902898 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900610 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 03:08:51.902898 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900613 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 03:08:51.902898 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900616 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 03:08:51.902898 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900618 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 03:08:51.902898 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900621 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 03:08:51.902898 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900623 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 03:08:51.902898 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900626 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 03:08:51.902898 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900628 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 03:08:51.902898 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900631 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 03:08:51.902898 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900633 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 03:08:51.902898 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900636 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 03:08:51.902898 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900638 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 03:08:51.902898 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900641 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 03:08:51.903385 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900643 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 03:08:51.903385 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900646 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 03:08:51.903385 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900648 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 03:08:51.903385 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900651 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 03:08:51.903385 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900653 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 03:08:51.903385 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900656 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 03:08:51.903385 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900659 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 03:08:51.903385 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900661 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 03:08:51.903385 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900664 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 03:08:51.903385 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900666 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 03:08:51.903385 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900669 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 03:08:51.903385 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900672 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 03:08:51.903385 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900674 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 03:08:51.903385 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900677 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 03:08:51.903385 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900679 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 03:08:51.903385 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900682 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 03:08:51.903385 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900684 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 03:08:51.903385 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900687 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 03:08:51.903385 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900691 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 03:08:51.903385 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900694 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 03:08:51.903884 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900697 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 15 03:08:51.903884 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900699 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 03:08:51.903884 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900702 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 03:08:51.903884 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900704 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 03:08:51.903884 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900707 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 03:08:51.903884 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900709 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 03:08:51.903884 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900712 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 03:08:51.903884 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900716 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 03:08:51.903884 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900719 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 03:08:51.903884 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900722 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 03:08:51.903884 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900726 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 03:08:51.903884 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900730 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 03:08:51.903884 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900733 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 03:08:51.903884 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900736 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 03:08:51.903884 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900739 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 03:08:51.903884 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900742 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 03:08:51.903884 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900744 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 03:08:51.903884 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900747 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 03:08:51.903884 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.900750 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 03:08:51.903884 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901868 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 03:08:51.904394 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901875 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 03:08:51.904394 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901879 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 03:08:51.904394 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901882 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 03:08:51.904394 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901885 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 03:08:51.904394 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901888 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 03:08:51.904394 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901891 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 03:08:51.904394 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901894 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 03:08:51.904394 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901897 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 03:08:51.904394 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901900 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 03:08:51.904394 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901903 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 03:08:51.904394 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901906 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 03:08:51.904394 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901909 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 03:08:51.904394 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901912 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 03:08:51.904394 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901915 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 03:08:51.904394 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901917 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 03:08:51.904394 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901920 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 03:08:51.904394 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901923 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 03:08:51.904394 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901926 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 03:08:51.904394 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901928 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 03:08:51.904394 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901931 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 03:08:51.904905 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901934 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 03:08:51.904905 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901936 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 03:08:51.904905 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901939 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 03:08:51.904905 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901942 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 03:08:51.904905 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901944 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 03:08:51.904905 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901947 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 03:08:51.904905 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901950 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 03:08:51.904905 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901952 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 03:08:51.904905 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901955 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 03:08:51.904905 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901958 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 03:08:51.904905 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901960 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 03:08:51.904905 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901963 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 03:08:51.904905 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901965 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 03:08:51.904905 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901968 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 03:08:51.904905 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901970 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 03:08:51.904905 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901973 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 03:08:51.904905 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901976 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 03:08:51.904905 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901979 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 03:08:51.904905 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901982 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 03:08:51.904905 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901984 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 03:08:51.905421 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901987 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 03:08:51.905421 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901990 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 03:08:51.905421 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901993 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 03:08:51.905421 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901995 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 03:08:51.905421 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.901998 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 03:08:51.905421 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902001 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 03:08:51.905421 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902004 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 03:08:51.905421 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902007 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 03:08:51.905421 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902010 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 03:08:51.905421 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902012 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 03:08:51.905421 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902015 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 03:08:51.905421 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902018 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 03:08:51.905421 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902020 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 03:08:51.905421 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902022 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 03:08:51.905421 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902025 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 03:08:51.905421 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902027 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 03:08:51.905421 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902030 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 03:08:51.905421 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902032 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 03:08:51.905421 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902035 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 03:08:51.905421 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902037 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 03:08:51.905957 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902040 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 03:08:51.905957 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902043 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 03:08:51.905957 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902045 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 15 03:08:51.905957 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902048 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 03:08:51.905957 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902050 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 03:08:51.905957 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902053 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 03:08:51.905957 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902055 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 03:08:51.905957 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902060 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 03:08:51.905957 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902063 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 03:08:51.905957 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902067 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 03:08:51.905957 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902070 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 03:08:51.905957 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902073 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 03:08:51.905957 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902076 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 03:08:51.905957 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902079 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 03:08:51.905957 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902082 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 03:08:51.905957 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902084 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 03:08:51.905957 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902087 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 03:08:51.905957 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902089 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 03:08:51.905957 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902092 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 03:08:51.905957 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902095 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 03:08:51.906436 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902097 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 03:08:51.906436 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902100 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 03:08:51.906436 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902102 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 03:08:51.906436 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902105 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 03:08:51.906436 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902107 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 03:08:51.906436 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902110 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 03:08:51.906436 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902113 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 03:08:51.906436 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902115 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 03:08:51.906436 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902118 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 03:08:51.906436 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902121 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 03:08:51.906436 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902123 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 03:08:51.906436 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902126 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 03:08:51.906436 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902128 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 03:08:51.906436 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902132 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 03:08:51.906436 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902135 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 03:08:51.906436 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902139 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 03:08:51.906436 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902144 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 03:08:51.906436 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902147 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 03:08:51.906436 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902212 2562 flags.go:64] FLAG: --address="0.0.0.0" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902220 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902227 2562 flags.go:64] FLAG: --anonymous-auth="true" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902231 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902236 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902239 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902243 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902247 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902250 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902253 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902257 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902260 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902263 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902266 2562 flags.go:64] FLAG: --cgroup-root="" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902269 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902275 2562 flags.go:64] FLAG: --client-ca-file="" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902277 2562 flags.go:64] FLAG: --cloud-provider="external" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902281 2562 flags.go:64] FLAG: --cluster-dns="[]" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902285 2562 flags.go:64] FLAG: --cluster-domain="" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902288 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902291 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902297 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902300 2562 flags.go:64] FLAG: --container-log-max-files="5" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902304 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 15 03:08:51.906951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902307 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902310 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902314 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902317 2562 flags.go:64] FLAG: --contention-profiling="false" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902319 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902322 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902325 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902328 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902332 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902335 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902338 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902341 2562 flags.go:64] FLAG: --enable-load-reader="false" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902344 2562 flags.go:64] FLAG: --enable-server="true" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902348 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902354 2562 flags.go:64] FLAG: --event-burst="100" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902357 2562 flags.go:64] FLAG: --event-qps="50" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902360 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902363 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902366 2562 flags.go:64] FLAG: --eviction-hard="" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902369 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902372 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902375 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902378 2562 flags.go:64] FLAG: --eviction-soft="" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902382 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902385 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 15 03:08:51.907545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902388 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902390 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902393 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902396 2562 flags.go:64] FLAG: --fail-swap-on="true" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902400 2562 flags.go:64] FLAG: --feature-gates="" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902404 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902407 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902411 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902414 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902417 2562 flags.go:64] FLAG: --healthz-port="10248" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902421 2562 flags.go:64] FLAG: --help="false" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902424 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-143-185.ec2.internal" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902427 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902430 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902432 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902436 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902439 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902442 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902445 2562 flags.go:64] FLAG: --image-service-endpoint="" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902448 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902451 2562 flags.go:64] FLAG: --kube-api-burst="100" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902454 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902457 2562 flags.go:64] FLAG: --kube-api-qps="50" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902460 2562 flags.go:64] FLAG: --kube-reserved="" Jun 15 03:08:51.908273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902463 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902466 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902469 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902471 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902474 2562 flags.go:64] FLAG: --lock-file="" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902477 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902481 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902484 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902489 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902492 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902495 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902498 2562 flags.go:64] FLAG: --logging-format="text" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902502 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902505 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902508 2562 flags.go:64] FLAG: --manifest-url="" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902511 2562 flags.go:64] FLAG: --manifest-url-header="" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902515 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902518 2562 flags.go:64] FLAG: --max-open-files="1000000" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902522 2562 flags.go:64] FLAG: --max-pods="110" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902525 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902528 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902531 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902534 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902536 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902539 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 15 03:08:51.908894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902542 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902549 2562 flags.go:64] FLAG: --node-status-max-images="50" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902552 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902555 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902558 2562 flags.go:64] FLAG: --pod-cidr="" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902561 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4a2e90c8ef7d783e4d635e24fb09435f5e1f6e954583512c0944f9e57bc77b52" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902566 2562 flags.go:64] FLAG: --pod-manifest-path="" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902569 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902572 2562 flags.go:64] FLAG: --pods-per-core="0" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902575 2562 flags.go:64] FLAG: --port="10250" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902578 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902581 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0361a7ff90f7e5f76" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902584 2562 flags.go:64] FLAG: --qos-reserved="" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902588 2562 flags.go:64] FLAG: --read-only-port="10255" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902591 2562 flags.go:64] FLAG: --register-node="true" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902594 2562 flags.go:64] FLAG: --register-with-taints="" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902598 2562 flags.go:64] FLAG: --registry-burst="10" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902601 2562 flags.go:64] FLAG: --registry-qps="5" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902605 2562 flags.go:64] FLAG: --reserved-cpus="" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902607 2562 flags.go:64] FLAG: --reserved-memory="" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902612 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902615 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902618 2562 flags.go:64] FLAG: --rotate-certificates="false" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902621 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902624 2562 flags.go:64] FLAG: --runonce="false" Jun 15 03:08:51.909504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902627 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902630 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902633 2562 flags.go:64] FLAG: --seccomp-default="false" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902636 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902639 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902642 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902645 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902648 2562 flags.go:64] FLAG: --storage-driver-password="root" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902651 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902654 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902657 2562 flags.go:64] FLAG: --storage-driver-user="root" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902659 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902662 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902666 2562 flags.go:64] FLAG: --system-cgroups="" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902668 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902674 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902677 2562 flags.go:64] FLAG: --tls-cert-file="" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902680 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902683 2562 flags.go:64] FLAG: --tls-min-version="" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902686 2562 flags.go:64] FLAG: --tls-private-key-file="" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902690 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902701 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902704 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902707 2562 flags.go:64] FLAG: --v="2" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902711 2562 flags.go:64] FLAG: --version="false" Jun 15 03:08:51.910123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902715 2562 flags.go:64] FLAG: --vmodule="" Jun 15 03:08:51.910780 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902719 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 15 03:08:51.910780 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.902724 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 15 03:08:51.910780 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902851 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 03:08:51.910780 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902856 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 15 03:08:51.910780 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902859 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 03:08:51.910780 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902862 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 03:08:51.910780 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902865 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 03:08:51.910780 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902868 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 03:08:51.910780 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902872 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 03:08:51.910780 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902875 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 03:08:51.910780 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902879 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 03:08:51.910780 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902882 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 03:08:51.910780 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902886 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 03:08:51.910780 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902889 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 03:08:51.910780 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902891 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 03:08:51.910780 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902894 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 03:08:51.910780 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902896 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 03:08:51.910780 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902899 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 03:08:51.910780 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902902 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 03:08:51.910780 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902904 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 03:08:51.911289 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902908 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 03:08:51.911289 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902910 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 03:08:51.911289 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902913 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 03:08:51.911289 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902916 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 03:08:51.911289 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902919 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 03:08:51.911289 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902921 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 03:08:51.911289 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902924 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 03:08:51.911289 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902927 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 03:08:51.911289 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902929 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 03:08:51.911289 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902932 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 03:08:51.911289 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902935 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 03:08:51.911289 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902937 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 03:08:51.911289 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902940 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 03:08:51.911289 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902943 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 03:08:51.911289 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902947 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 03:08:51.911289 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902951 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 03:08:51.911289 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902955 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 03:08:51.911289 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902958 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 03:08:51.911725 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902961 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 03:08:51.911725 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902964 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 03:08:51.911725 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902967 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 03:08:51.911725 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902969 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 03:08:51.911725 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902972 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 03:08:51.911725 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902975 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 03:08:51.911725 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902978 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 03:08:51.911725 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902981 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 03:08:51.911725 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902984 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 03:08:51.911725 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902986 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 03:08:51.911725 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902989 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 03:08:51.911725 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902992 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 03:08:51.911725 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902995 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 03:08:51.911725 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.902998 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 03:08:51.911725 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903000 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 03:08:51.911725 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903003 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 03:08:51.911725 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903006 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 03:08:51.911725 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903008 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 03:08:51.911725 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903011 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 03:08:51.911725 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903013 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 03:08:51.912240 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903016 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 03:08:51.912240 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903019 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 03:08:51.912240 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903021 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 03:08:51.912240 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903024 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 03:08:51.912240 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903026 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 03:08:51.912240 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903028 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 03:08:51.912240 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903031 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 03:08:51.912240 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903034 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 03:08:51.912240 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903036 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 03:08:51.912240 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903039 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 03:08:51.912240 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903041 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 03:08:51.912240 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903044 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 03:08:51.912240 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903047 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 03:08:51.912240 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903049 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 03:08:51.912240 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903052 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 03:08:51.912240 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903054 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 03:08:51.912240 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903056 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 03:08:51.912240 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903059 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 03:08:51.912240 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903062 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 03:08:51.912240 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903064 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 03:08:51.912724 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903067 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 03:08:51.912724 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903069 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 03:08:51.912724 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903072 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 03:08:51.912724 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903074 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 03:08:51.912724 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903077 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 03:08:51.912724 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903079 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 03:08:51.912724 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903082 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 03:08:51.912724 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903084 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 03:08:51.912724 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903088 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 03:08:51.912724 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903090 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 03:08:51.912724 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903093 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 03:08:51.912724 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903095 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 03:08:51.912724 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903098 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 03:08:51.912724 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903101 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 03:08:51.912724 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903103 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 03:08:51.912724 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903106 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 03:08:51.912724 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903108 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 03:08:51.912724 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903111 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 03:08:51.912724 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903113 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 03:08:51.912724 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903116 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 03:08:51.913271 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903118 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 03:08:51.913271 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903121 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 03:08:51.913271 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.903123 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 03:08:51.913271 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.903132 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 15 03:08:51.913271 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.909768 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 15 03:08:51.913271 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.909782 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 15 03:08:51.913271 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.909801 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 15 03:08:51.913271 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.909807 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 15 03:08:51.913271 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909892 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 03:08:51.913271 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909899 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 03:08:51.913271 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909903 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 03:08:51.913271 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909907 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 03:08:51.913271 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909910 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 03:08:51.913271 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909914 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 03:08:51.913626 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909916 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 03:08:51.913626 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909926 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 03:08:51.913626 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909929 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 03:08:51.913626 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909932 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 03:08:51.913626 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909938 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 03:08:51.913626 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909941 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 03:08:51.913626 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909949 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 03:08:51.913626 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909952 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 03:08:51.913626 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909955 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 03:08:51.913626 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909958 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 03:08:51.913626 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909962 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 03:08:51.913626 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909965 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 03:08:51.913626 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909968 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 03:08:51.913626 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909971 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 03:08:51.913626 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909973 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 03:08:51.913626 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909976 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 03:08:51.913626 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909978 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 03:08:51.913626 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909981 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 03:08:51.913626 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909984 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 03:08:51.914098 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909986 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 03:08:51.914098 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909989 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 03:08:51.914098 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909991 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 03:08:51.914098 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909994 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 03:08:51.914098 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.909997 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 03:08:51.914098 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910000 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 03:08:51.914098 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910003 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 03:08:51.914098 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910005 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 03:08:51.914098 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910008 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 15 03:08:51.914098 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910011 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 03:08:51.914098 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910013 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 03:08:51.914098 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910016 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 03:08:51.914098 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910019 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 03:08:51.914098 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910022 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 03:08:51.914098 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910026 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 03:08:51.914098 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910029 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 03:08:51.914098 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910033 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 03:08:51.914098 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910035 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 03:08:51.914098 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910038 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 03:08:51.914098 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910041 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 03:08:51.914587 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910049 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 03:08:51.914587 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910052 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 03:08:51.914587 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910055 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 03:08:51.914587 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910057 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 03:08:51.914587 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910060 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 03:08:51.914587 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910062 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 03:08:51.914587 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910065 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 03:08:51.914587 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910068 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 03:08:51.914587 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910070 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 03:08:51.914587 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910072 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 03:08:51.914587 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910075 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 03:08:51.914587 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910078 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 03:08:51.914587 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910080 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 03:08:51.914587 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910083 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 03:08:51.914587 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910086 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 03:08:51.914587 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910088 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 03:08:51.914587 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910093 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 03:08:51.914587 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910096 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 03:08:51.914587 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910098 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 03:08:51.914587 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910101 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 03:08:51.915092 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910103 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 03:08:51.915092 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910106 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 03:08:51.915092 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910108 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 03:08:51.915092 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910111 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 03:08:51.915092 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910113 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 03:08:51.915092 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910116 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 03:08:51.915092 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910118 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 03:08:51.915092 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910121 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 03:08:51.915092 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910123 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 03:08:51.915092 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910126 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 03:08:51.915092 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910128 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 03:08:51.915092 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910131 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 03:08:51.915092 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910134 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 03:08:51.915092 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910143 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 03:08:51.915092 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910146 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 03:08:51.915092 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910148 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 03:08:51.915092 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910151 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 03:08:51.915092 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910153 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 03:08:51.915092 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910156 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 03:08:51.915092 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910158 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 03:08:51.915585 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910161 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 03:08:51.915585 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910163 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 03:08:51.915585 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910166 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 03:08:51.915585 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910168 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 03:08:51.915585 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910170 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 03:08:51.915585 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910173 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 03:08:51.915585 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910175 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 03:08:51.915585 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910178 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 03:08:51.915585 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910181 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 03:08:51.915585 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910184 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 03:08:51.915585 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910187 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 03:08:51.915585 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910189 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 03:08:51.915585 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910192 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 03:08:51.915585 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910194 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 03:08:51.915946 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.910199 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 15 03:08:51.915946 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910356 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 15 03:08:51.915946 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910361 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 15 03:08:51.915946 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910364 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 15 03:08:51.915946 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910367 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 15 03:08:51.915946 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910370 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 15 03:08:51.915946 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910372 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 15 03:08:51.915946 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910376 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 15 03:08:51.915946 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910380 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 15 03:08:51.915946 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910383 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 15 03:08:51.915946 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910386 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 15 03:08:51.915946 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910388 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 15 03:08:51.915946 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910397 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 15 03:08:51.915946 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910399 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 15 03:08:51.916290 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910402 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 15 03:08:51.916290 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910405 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 15 03:08:51.916290 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910407 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 15 03:08:51.916290 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910409 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 15 03:08:51.916290 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910412 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 15 03:08:51.916290 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910414 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 15 03:08:51.916290 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910417 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 15 03:08:51.916290 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910419 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 15 03:08:51.916290 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910422 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 15 03:08:51.916290 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910425 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 15 03:08:51.916290 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910428 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 15 03:08:51.916290 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910430 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 15 03:08:51.916290 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910433 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 15 03:08:51.916290 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910436 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 15 03:08:51.916290 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910438 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 15 03:08:51.916290 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910441 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 15 03:08:51.916290 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910443 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 15 03:08:51.916290 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910446 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 15 03:08:51.916290 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910448 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 15 03:08:51.916290 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910451 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 15 03:08:51.916776 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910453 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 15 03:08:51.916776 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910456 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 15 03:08:51.916776 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910458 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 15 03:08:51.916776 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910461 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 15 03:08:51.916776 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910463 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 15 03:08:51.916776 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910466 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 15 03:08:51.916776 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910468 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 15 03:08:51.916776 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910471 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 15 03:08:51.916776 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910473 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 15 03:08:51.916776 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910476 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 15 03:08:51.916776 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910478 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 15 03:08:51.916776 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910486 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 15 03:08:51.916776 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910489 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 15 03:08:51.916776 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910491 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 15 03:08:51.916776 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910494 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 15 03:08:51.916776 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910496 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 15 03:08:51.916776 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910499 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 15 03:08:51.916776 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910501 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 15 03:08:51.916776 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910504 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 15 03:08:51.916776 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910506 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 15 03:08:51.917261 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910508 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 15 03:08:51.917261 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910511 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 15 03:08:51.917261 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910513 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 15 03:08:51.917261 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910516 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 15 03:08:51.917261 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910519 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 15 03:08:51.917261 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910521 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 15 03:08:51.917261 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910524 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 15 03:08:51.917261 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910527 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 15 03:08:51.917261 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910530 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 15 03:08:51.917261 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910532 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 15 03:08:51.917261 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910535 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 15 03:08:51.917261 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910538 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 15 03:08:51.917261 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910540 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 15 03:08:51.917261 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910543 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 15 03:08:51.917261 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910545 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 15 03:08:51.917261 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910548 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 15 03:08:51.917261 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910550 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 15 03:08:51.917261 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910553 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 15 03:08:51.917261 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910555 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 15 03:08:51.917713 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910557 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 15 03:08:51.917713 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910560 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 15 03:08:51.917713 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910563 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 15 03:08:51.917713 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910565 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 15 03:08:51.917713 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910568 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 15 03:08:51.917713 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910575 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 15 03:08:51.917713 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910578 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 15 03:08:51.917713 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910582 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 15 03:08:51.917713 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910585 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 15 03:08:51.917713 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910588 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 15 03:08:51.917713 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910591 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 15 03:08:51.917713 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910593 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 15 03:08:51.917713 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910596 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 15 03:08:51.917713 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910599 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 15 03:08:51.917713 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910601 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 15 03:08:51.917713 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910604 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 15 03:08:51.917713 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910608 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 15 03:08:51.917713 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910611 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 15 03:08:51.917713 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910613 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 15 03:08:51.917713 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910616 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 15 03:08:51.918208 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910618 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 15 03:08:51.918208 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910621 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 15 03:08:51.918208 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910623 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 15 03:08:51.918208 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910626 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 15 03:08:51.918208 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910628 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 15 03:08:51.918208 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910631 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 15 03:08:51.918208 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:51.910635 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 15 03:08:51.918208 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.910639 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 15 03:08:51.918208 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.910758 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jun 15 03:08:51.918208 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.914491 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 15 03:08:51.918208 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.915582 2562 server.go:1019] "Starting client certificate rotation" Jun 15 03:08:51.918208 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.915678 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 15 03:08:51.918208 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.915720 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 15 03:08:51.918691 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.917622 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 15 03:08:51.921287 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.921267 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 15 03:08:51.930064 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.930044 2562 log.go:25] "Validated CRI v1 runtime API" Jun 15 03:08:51.932803 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.932780 2562 log.go:25] "Validated CRI v1 image API" Jun 15 03:08:51.935140 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.935118 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 15 03:08:51.937745 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.937722 2562 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9624e78b-4d1e-432d-9134-7ba2bafbf80e:/dev/nvme0n1p4 d03c5e60-fc26-4a99-8cb3-d9f440e9c625:/dev/nvme0n1p3] Jun 15 03:08:51.937820 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.937745 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 15 03:08:51.942870 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.942665 2562 manager.go:216] Machine: {Timestamp:2026-06-15 03:08:51.94149618 +0000 UTC m=+0.252635161 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099643 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2849ec9277dec9b16b4dcf3c081721 SystemUUID:ec2849ec-9277-dec9-b16b-4dcf3c081721 BootID:4bc0d905-d84d-446d-9687-3d3e91ddb922 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1f:f7:80:14:4f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1f:f7:80:14:4f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:76:d9:eb:75:67:4f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 15 03:08:51.942870 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.942826 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 15 03:08:51.943038 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.942966 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.120.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260608-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 15 03:08:51.945252 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.945218 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 15 03:08:51.945424 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.945409 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 15 03:08:51.945491 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.945255 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-185.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 15 03:08:51.946403 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.946391 2562 topology_manager.go:138] "Creating topology manager with none policy" Jun 15 03:08:51.946466 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.946411 2562 container_manager_linux.go:309] "Creating device plugin manager" Jun 15 03:08:51.946466 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.946422 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 15 03:08:51.946964 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.946952 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 15 03:08:51.948100 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.948088 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 15 03:08:51.951770 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.951756 2562 state_mem.go:36] "Initialized new in-memory state store" Jun 15 03:08:51.951916 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.951905 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 15 03:08:51.951984 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.951943 2562 kubelet.go:486] "Attempting to sync node with API server" Jun 15 03:08:51.951984 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.951960 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 15 03:08:51.951984 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.951976 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 15 03:08:51.952106 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.951989 2562 kubelet.go:398] "Adding apiserver pod source" Jun 15 03:08:51.952106 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.952000 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 15 03:08:51.952420 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.952379 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 15 03:08:51.952420 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.952401 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 15 03:08:51.954172 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.954154 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 15 03:08:51.955956 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.955931 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 15 03:08:51.956022 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.955969 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 15 03:08:51.956578 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.956565 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 15 03:08:51.956613 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.956589 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 15 03:08:51.956613 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.956599 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 15 03:08:51.956613 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.956608 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 15 03:08:51.956686 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.956626 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 15 03:08:51.956686 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.956639 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 15 03:08:51.956686 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.956650 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 15 03:08:51.956686 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.956657 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 15 03:08:51.956686 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.956669 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 15 03:08:51.956686 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.956675 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 15 03:08:51.956686 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.956683 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 15 03:08:51.956686 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.956688 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 15 03:08:51.957318 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.957309 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 15 03:08:51.957318 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.957317 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 15 03:08:51.960494 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.960472 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-185.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 15 03:08:51.960561 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:51.960509 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-185.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 15 03:08:51.961125 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.961099 2562 server.go:1268] "Started kubelet" Jun 15 03:08:51.962244 ip-10-0-143-185 systemd[1]: Started Kubernetes Kubelet. Jun 15 03:08:51.962373 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:51.960991 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 15 03:08:51.962455 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.962389 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 15 03:08:51.962522 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.962403 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 15 03:08:51.962641 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.962620 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 15 03:08:51.964976 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.964959 2562 server.go:310] "Adding debug handlers to kubelet server" Jun 15 03:08:51.965641 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.965620 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 15 03:08:51.966675 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.966654 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nn5jf" Jun 15 03:08:51.968943 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:51.968922 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 15 03:08:51.969095 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:51.968356 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-185.ec2.internal.18b922620368156c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-185.ec2.internal,UID:ip-10-0-143-185.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-185.ec2.internal,},FirstTimestamp:2026-06-15 03:08:51.961034092 +0000 UTC m=+0.272173078,LastTimestamp:2026-06-15 03:08:51.961034092 +0000 UTC m=+0.272173078,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-185.ec2.internal,}" Jun 15 03:08:51.969925 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.969906 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 15 03:08:51.969925 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.969912 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 15 03:08:51.970055 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:51.970006 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jun 15 03:08:51.970500 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.970483 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 15 03:08:51.970500 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.970485 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 15 03:08:51.970628 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.970512 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 15 03:08:51.970628 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.970620 2562 reconstruct.go:97] "Volume reconstruction finished" Jun 15 03:08:51.970628 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.970628 2562 reconciler.go:29] "Reconciler: start to sync state" Jun 15 03:08:51.970780 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.970658 2562 factory.go:55] Registering systemd factory Jun 15 03:08:51.970780 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.970672 2562 factory.go:223] Registration of the systemd container factory successfully Jun 15 03:08:51.970938 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.970913 2562 factory.go:153] Registering CRI-O factory Jun 15 03:08:51.970938 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.970932 2562 factory.go:223] Registration of the crio container factory successfully Jun 15 03:08:51.971115 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.970983 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 15 03:08:51.971115 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.971007 2562 factory.go:103] Registering Raw factory Jun 15 03:08:51.971115 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.971021 2562 manager.go:1196] Started watching for new ooms in manager Jun 15 03:08:51.971471 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.971459 2562 manager.go:349] Starting recovery of all containers Jun 15 03:08:51.973242 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:51.973213 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 15 03:08:51.973320 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:51.973236 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-185.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 15 03:08:51.974888 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.974867 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nn5jf" Jun 15 03:08:51.981049 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.980895 2562 manager.go:354] Recovery completed Jun 15 03:08:51.986262 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.986248 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 03:08:51.988191 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.988176 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientMemory" Jun 15 03:08:51.988268 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.988202 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 03:08:51.988268 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.988213 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientPID" Jun 15 03:08:51.988713 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.988698 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 15 03:08:51.988713 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.988713 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 15 03:08:51.988863 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.988715 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 03:08:51.988863 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.988729 2562 state_mem.go:36] "Initialized new in-memory state store" Jun 15 03:08:51.989490 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.989477 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientMemory" Jun 15 03:08:51.989554 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.989504 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 03:08:51.989554 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.989518 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientPID" Jun 15 03:08:51.990942 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.990920 2562 policy_none.go:49] "None policy: Start" Jun 15 03:08:51.991018 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.990946 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 15 03:08:51.991018 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.990960 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 15 03:08:51.991400 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:51.991286 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-185.ec2.internal.18b922620506746f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-185.ec2.internal,UID:ip-10-0-143-185.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-185.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-185.ec2.internal,},FirstTimestamp:2026-06-15 03:08:51.988190319 +0000 UTC m=+0.299329299,LastTimestamp:2026-06-15 03:08:51.988190319 +0000 UTC m=+0.299329299,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-185.ec2.internal,}" Jun 15 03:08:51.993099 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:51.993084 2562 policy_none.go:47] "Start" Jun 15 03:08:52.031475 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.031456 2562 manager.go:337] "Starting Device Plugin manager" Jun 15 03:08:52.052674 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:52.031499 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 15 03:08:52.052674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.031512 2562 server.go:87] "Starting device plugin registration server" Jun 15 03:08:52.052674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.031806 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 15 03:08:52.052674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.031821 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 15 03:08:52.052674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.031962 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 15 03:08:52.052674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.032017 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 15 03:08:52.052674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.032033 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 15 03:08:52.052674 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:52.032521 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 15 03:08:52.052674 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:52.032565 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-185.ec2.internal\" not found" Jun 15 03:08:52.052674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.037742 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 15 03:08:52.052674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.039073 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 15 03:08:52.052674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.039104 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 15 03:08:52.052674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.039123 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jun 15 03:08:52.052674 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:52.039152 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 15 03:08:52.052674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.041364 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 03:08:52.132792 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.132758 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 03:08:52.133911 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.133893 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientMemory" Jun 15 03:08:52.133988 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.133925 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 03:08:52.133988 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.133937 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientPID" Jun 15 03:08:52.133988 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.133961 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-185.ec2.internal" Jun 15 03:08:52.140241 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.140223 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal"] Jun 15 03:08:52.140296 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.140283 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 03:08:52.141476 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.141462 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientMemory" Jun 15 03:08:52.141536 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.141488 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 03:08:52.141536 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.141500 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientPID" Jun 15 03:08:52.141813 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.141800 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-185.ec2.internal" Jun 15 03:08:52.141865 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:52.141819 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-185.ec2.internal\": node \"ip-10-0-143-185.ec2.internal\" not found" Jun 15 03:08:52.143802 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.143790 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 03:08:52.143926 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.143911 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" Jun 15 03:08:52.143966 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.143943 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 03:08:52.144541 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.144491 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientMemory" Jun 15 03:08:52.144541 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.144516 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 03:08:52.144541 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.144527 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientPID" Jun 15 03:08:52.144672 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.144562 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientMemory" Jun 15 03:08:52.144672 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.144586 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 03:08:52.144672 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.144599 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientPID" Jun 15 03:08:52.146722 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.146708 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal" Jun 15 03:08:52.146812 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.146733 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 15 03:08:52.147384 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.147369 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientMemory" Jun 15 03:08:52.147434 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.147397 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasNoDiskPressure" Jun 15 03:08:52.147434 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.147407 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientPID" Jun 15 03:08:52.154431 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:52.154417 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jun 15 03:08:52.172470 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.172448 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1cd3eb1948b1e36499470a2dffb1c745-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal\" (UID: \"1cd3eb1948b1e36499470a2dffb1c745\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" Jun 15 03:08:52.172554 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.172474 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1cd3eb1948b1e36499470a2dffb1c745-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal\" (UID: \"1cd3eb1948b1e36499470a2dffb1c745\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" Jun 15 03:08:52.172554 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.172492 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a43d74820b080a54928ef60318b8a3ec-config\") pod \"kube-apiserver-proxy-ip-10-0-143-185.ec2.internal\" (UID: \"a43d74820b080a54928ef60318b8a3ec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal" Jun 15 03:08:52.178659 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:52.178644 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-185.ec2.internal\" not found" node="ip-10-0-143-185.ec2.internal" Jun 15 03:08:52.182928 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:52.182914 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-185.ec2.internal\" not found" node="ip-10-0-143-185.ec2.internal" Jun 15 03:08:52.255294 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:52.255261 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jun 15 03:08:52.273459 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.273438 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1cd3eb1948b1e36499470a2dffb1c745-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal\" (UID: \"1cd3eb1948b1e36499470a2dffb1c745\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" Jun 15 03:08:52.273545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.273464 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1cd3eb1948b1e36499470a2dffb1c745-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal\" (UID: \"1cd3eb1948b1e36499470a2dffb1c745\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" Jun 15 03:08:52.273545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.273482 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a43d74820b080a54928ef60318b8a3ec-config\") pod \"kube-apiserver-proxy-ip-10-0-143-185.ec2.internal\" (UID: \"a43d74820b080a54928ef60318b8a3ec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal" Jun 15 03:08:52.273545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.273520 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a43d74820b080a54928ef60318b8a3ec-config\") pod \"kube-apiserver-proxy-ip-10-0-143-185.ec2.internal\" (UID: \"a43d74820b080a54928ef60318b8a3ec\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal" Jun 15 03:08:52.273693 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.273545 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1cd3eb1948b1e36499470a2dffb1c745-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal\" (UID: \"1cd3eb1948b1e36499470a2dffb1c745\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" Jun 15 03:08:52.273693 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.273548 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1cd3eb1948b1e36499470a2dffb1c745-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal\" (UID: \"1cd3eb1948b1e36499470a2dffb1c745\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" Jun 15 03:08:52.356100 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:52.356054 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jun 15 03:08:52.456887 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:52.456807 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jun 15 03:08:52.481269 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.481235 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" Jun 15 03:08:52.486001 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.485713 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal" Jun 15 03:08:52.556989 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:52.556952 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jun 15 03:08:52.657453 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:52.657425 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jun 15 03:08:52.757987 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:52.757900 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jun 15 03:08:52.858425 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:52.858394 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jun 15 03:08:52.915828 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.915799 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 15 03:08:52.916371 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.915961 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 15 03:08:52.959261 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:52.959229 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jun 15 03:08:52.980104 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.980068 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-14 03:03:51 +0000 UTC" deadline="2028-02-28 19:29:53.793972928 +0000 UTC" Jun 15 03:08:52.980104 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:52.980096 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14968h21m0.813880871s" Jun 15 03:08:53.024275 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:53.024241 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cd3eb1948b1e36499470a2dffb1c745.slice/crio-fbac6914b703f68b999d880c86aa2bbf23e80e32b37415acfa526d14be8666a4 WatchSource:0}: Error finding container fbac6914b703f68b999d880c86aa2bbf23e80e32b37415acfa526d14be8666a4: Status 404 returned error can't find the container with id fbac6914b703f68b999d880c86aa2bbf23e80e32b37415acfa526d14be8666a4 Jun 15 03:08:53.025011 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:53.024985 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda43d74820b080a54928ef60318b8a3ec.slice/crio-5e2ab7eccdcf3ea1001923cbf715c2d8a9e7b58235d1f82e2ba349f1ba0fcd53 WatchSource:0}: Error finding container 5e2ab7eccdcf3ea1001923cbf715c2d8a9e7b58235d1f82e2ba349f1ba0fcd53: Status 404 returned error can't find the container with id 5e2ab7eccdcf3ea1001923cbf715c2d8a9e7b58235d1f82e2ba349f1ba0fcd53 Jun 15 03:08:53.028763 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.028745 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 15 03:08:53.042384 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.042341 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal" event={"ID":"a43d74820b080a54928ef60318b8a3ec","Type":"ContainerStarted","Data":"5e2ab7eccdcf3ea1001923cbf715c2d8a9e7b58235d1f82e2ba349f1ba0fcd53"} Jun 15 03:08:53.043459 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.043438 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" event={"ID":"1cd3eb1948b1e36499470a2dffb1c745","Type":"ContainerStarted","Data":"fbac6914b703f68b999d880c86aa2bbf23e80e32b37415acfa526d14be8666a4"} Jun 15 03:08:53.059595 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:53.059570 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jun 15 03:08:53.160109 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:53.160078 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jun 15 03:08:53.260664 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:53.260626 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jun 15 03:08:53.279510 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.278579 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 03:08:53.360815 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:53.360779 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jun 15 03:08:53.461213 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:53.461166 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jun 15 03:08:53.501935 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.501826 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 03:08:53.515315 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.515285 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 03:08:53.571129 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.570972 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" Jun 15 03:08:53.581971 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.581946 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 15 03:08:53.582786 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.582764 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal" Jun 15 03:08:53.591168 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.591072 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 15 03:08:53.953471 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.953327 2562 apiserver.go:52] "Watching apiserver" Jun 15 03:08:53.960629 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.960597 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 15 03:08:53.961769 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.961739 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-9r9mn","kube-system/konnectivity-agent-s4gh2","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z","openshift-dns/node-resolver-lj57j","openshift-image-registry/node-ca-xhccz","openshift-multus/multus-additional-cni-plugins-pn582","openshift-multus/multus-tpvc8","openshift-network-operator/iptables-alerter-ljsf5","kube-system/global-pull-secret-syncer-mbtvm","kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal","openshift-cluster-node-tuning-operator/tuned-t98lm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal","openshift-multus/network-metrics-daemon-vz9g7","openshift-network-diagnostics/network-check-target-nhnmr"] Jun 15 03:08:53.964600 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.964573 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tpvc8" Jun 15 03:08:53.966762 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.966715 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:08:53.966869 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:53.966802 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vz9g7" podUID="c85b7d1f-4d4b-4fdd-9427-d1c25942aebe" Jun 15 03:08:53.967251 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.967104 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 15 03:08:53.967251 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.967130 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 15 03:08:53.967251 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.967206 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 15 03:08:53.967251 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.967218 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hl9q5\"" Jun 15 03:08:53.967484 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.967421 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 15 03:08:53.969453 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.969431 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:53.970755 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.970736 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 15 03:08:53.971118 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.971095 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nb6z4\"" Jun 15 03:08:53.971246 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.971170 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 15 03:08:53.971636 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.971430 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lj57j" Jun 15 03:08:53.971721 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.971690 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 15 03:08:53.971827 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.971807 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 15 03:08:53.973214 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.973190 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6zjqj\"" Jun 15 03:08:53.973468 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.973416 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 15 03:08:53.973468 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.973417 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 15 03:08:53.973679 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.973658 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xhccz" Jun 15 03:08:53.975445 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.975293 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 15 03:08:53.975445 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.975346 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 15 03:08:53.975607 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.975445 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 15 03:08:53.975607 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.975454 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fq5tt\"" Jun 15 03:08:53.976162 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.975934 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:53.977924 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.977847 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 15 03:08:53.978320 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.978109 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-69lfj\"" Jun 15 03:08:53.978559 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.978541 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 15 03:08:53.979194 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.979093 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:08:53.979194 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:53.979161 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mbtvm" podUID="a3db17e5-7acd-462b-a6f2-b11617619628" Jun 15 03:08:53.980911 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.980891 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 15 03:08:53.983361 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.982525 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:53.983699 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.983674 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-system-cni-dir\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:53.983788 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.983712 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-os-release\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:53.983788 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.983744 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-host-var-lib-kubelet\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:53.983788 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.983776 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d75956ab-4288-4724-a60e-f55e2cb6c932-multus-daemon-config\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:53.983963 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.983805 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5khg7\" (UniqueName: \"kubernetes.io/projected/d75956ab-4288-4724-a60e-f55e2cb6c932-kube-api-access-5khg7\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:53.983963 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.983853 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/80ab5903-b44d-4517-8c0e-fe4abf44d298-etc-selinux\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:53.983963 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.983878 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4695v\" (UniqueName: \"kubernetes.io/projected/80ab5903-b44d-4517-8c0e-fe4abf44d298-kube-api-access-4695v\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:53.983963 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.983912 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-multus-socket-dir-parent\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:53.984156 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.983957 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-host-var-lib-cni-multus\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:53.984156 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.984013 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/80ab5903-b44d-4517-8c0e-fe4abf44d298-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:53.984156 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.984046 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/80ab5903-b44d-4517-8c0e-fe4abf44d298-socket-dir\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:53.986246 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.984074 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/80ab5903-b44d-4517-8c0e-fe4abf44d298-registration-dir\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:53.986246 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.984615 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-host-var-lib-cni-bin\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:53.986246 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.984634 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s4gh2" Jun 15 03:08:53.986246 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.984653 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-etc-kubernetes\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:53.986246 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.984686 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5d9r\" (UniqueName: \"kubernetes.io/projected/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-kube-api-access-s5d9r\") pod \"network-metrics-daemon-vz9g7\" (UID: \"c85b7d1f-4d4b-4fdd-9427-d1c25942aebe\") " pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:08:53.986246 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.984720 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/80ab5903-b44d-4517-8c0e-fe4abf44d298-device-dir\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:53.986246 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.984744 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/80ab5903-b44d-4517-8c0e-fe4abf44d298-sys-fs\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:53.986246 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.984780 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-multus-cni-dir\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:53.986246 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.984812 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs\") pod \"network-metrics-daemon-vz9g7\" (UID: \"c85b7d1f-4d4b-4fdd-9427-d1c25942aebe\") " pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:08:53.986246 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.984860 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff9255c7-dc08-4be9-9087-b749284c6358-tmp-dir\") pod \"node-resolver-lj57j\" (UID: \"ff9255c7-dc08-4be9-9087-b749284c6358\") " pod="openshift-dns/node-resolver-lj57j" Jun 15 03:08:53.986246 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.984894 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d75956ab-4288-4724-a60e-f55e2cb6c932-cni-binary-copy\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:53.986246 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.984923 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-host-run-netns\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:53.986246 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.984957 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-multus-conf-dir\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:53.986246 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.984986 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ff9255c7-dc08-4be9-9087-b749284c6358-hosts-file\") pod \"node-resolver-lj57j\" (UID: \"ff9255c7-dc08-4be9-9087-b749284c6358\") " pod="openshift-dns/node-resolver-lj57j" Jun 15 03:08:53.986246 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.985018 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-cnibin\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:53.986246 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.985053 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-host-run-multus-certs\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:53.986246 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.985084 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-549n9\" (UniqueName: \"kubernetes.io/projected/ff9255c7-dc08-4be9-9087-b749284c6358-kube-api-access-549n9\") pod \"node-resolver-lj57j\" (UID: \"ff9255c7-dc08-4be9-9087-b749284c6358\") " pod="openshift-dns/node-resolver-lj57j" Jun 15 03:08:53.987183 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.985134 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-host-run-k8s-cni-cncf-io\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:53.987183 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.985166 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-hostroot\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:53.987183 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.985443 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 15 03:08:53.987183 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.985554 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 15 03:08:53.987183 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.985618 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 15 03:08:53.987183 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.985691 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 15 03:08:53.987183 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.985864 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-48zxw\"" Jun 15 03:08:53.987183 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.985909 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 15 03:08:53.987183 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.986096 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 15 03:08:53.988027 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.987556 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 15 03:08:53.988027 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.987633 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-m48wh\"" Jun 15 03:08:53.988027 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.987673 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 15 03:08:53.988027 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.987864 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:08:53.988027 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:53.987967 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nhnmr" podUID="5488015e-fad7-4f27-b7a0-62d9bb3d7da0" Jun 15 03:08:53.990114 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.990097 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:53.992025 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.991985 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 15 03:08:53.992025 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.991988 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zrjq6\"" Jun 15 03:08:53.992166 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.992068 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 15 03:08:53.992540 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.992520 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ljsf5" Jun 15 03:08:53.995058 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.994723 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 15 03:08:53.995058 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.994743 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 15 03:08:53.995058 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.994773 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 15 03:08:53.995058 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:53.994725 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hdx9v\"" Jun 15 03:08:54.016546 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.016527 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6pnbs" Jun 15 03:08:54.023627 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.023604 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6pnbs" Jun 15 03:08:54.072231 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.072204 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 15 03:08:54.085517 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.085492 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0a161380-ecd7-4473-9cc2-e3041ad5571d-konnectivity-ca\") pod \"konnectivity-agent-s4gh2\" (UID: \"0a161380-ecd7-4473-9cc2-e3041ad5571d\") " pod="kube-system/konnectivity-agent-s4gh2" Jun 15 03:08:54.085674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.085522 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-systemd\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.085674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.085538 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-var-lib-kubelet\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.085674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.085577 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-kubelet\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.085674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.085596 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-systemd-units\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.085674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.085624 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjjrr\" (UniqueName: \"kubernetes.io/projected/795cf261-afbb-4f3f-84a1-617bc13d25f9-kube-api-access-gjjrr\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.085674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.085649 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr2g5\" (UniqueName: \"kubernetes.io/projected/7fe3e8d4-b5da-4511-a7f8-f28596c5f336-kube-api-access-tr2g5\") pod \"node-ca-xhccz\" (UID: \"7fe3e8d4-b5da-4511-a7f8-f28596c5f336\") " pod="openshift-image-registry/node-ca-xhccz" Jun 15 03:08:54.085674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.085668 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-sysctl-d\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.085996 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.085706 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-tuned\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.085996 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.085737 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/28f12465-f135-4586-a19f-98430d1f26b0-os-release\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.085996 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.085767 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/28f12465-f135-4586-a19f-98430d1f26b0-cni-binary-copy\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.085996 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.085793 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/28f12465-f135-4586-a19f-98430d1f26b0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.085996 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.085821 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-run-ovn-kubernetes\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.085996 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.085876 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-cnibin\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.085996 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.085906 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-sysconfig\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.085996 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.085933 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-etc-openvswitch\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.085996 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.085986 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-cnibin\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.085996 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.085988 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.086415 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086027 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/789deb33-3a27-4d7f-881f-31f889f46e0e-ovn-node-metrics-cert\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.086415 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086073 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret\") pod \"global-pull-secret-syncer-mbtvm\" (UID: \"a3db17e5-7acd-462b-a6f2-b11617619628\") " pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:08:54.086415 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086118 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-log-socket\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.086415 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086150 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-cni-bin\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.086415 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086191 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5pnn\" (UniqueName: \"kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn\") pod \"network-check-target-nhnmr\" (UID: \"5488015e-fad7-4f27-b7a0-62d9bb3d7da0\") " pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:08:54.086415 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086217 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-os-release\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.086415 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086237 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d75956ab-4288-4724-a60e-f55e2cb6c932-multus-daemon-config\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.086415 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086256 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4695v\" (UniqueName: \"kubernetes.io/projected/80ab5903-b44d-4517-8c0e-fe4abf44d298-kube-api-access-4695v\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:54.086415 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086278 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0a161380-ecd7-4473-9cc2-e3041ad5571d-agent-certs\") pod \"konnectivity-agent-s4gh2\" (UID: \"0a161380-ecd7-4473-9cc2-e3041ad5571d\") " pod="kube-system/konnectivity-agent-s4gh2" Jun 15 03:08:54.086415 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086294 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-sysctl-conf\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.086415 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086312 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-os-release\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.086860 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086410 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/789deb33-3a27-4d7f-881f-31f889f46e0e-env-overrides\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.086860 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086465 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-host-var-lib-cni-multus\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.086860 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086499 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-host-var-lib-cni-multus\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.086860 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086540 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/80ab5903-b44d-4517-8c0e-fe4abf44d298-socket-dir\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:54.086860 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086579 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/80ab5903-b44d-4517-8c0e-fe4abf44d298-registration-dir\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:54.086860 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086625 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-node-log\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.086860 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086631 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/80ab5903-b44d-4517-8c0e-fe4abf44d298-socket-dir\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:54.086860 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086653 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/80ab5903-b44d-4517-8c0e-fe4abf44d298-registration-dir\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:54.086860 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086684 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-lib-modules\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.086860 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086719 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-slash\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.087274 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.086760 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-var-lib-openvswitch\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.087324 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.087270 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-run-openvswitch\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.087324 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.087289 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d75956ab-4288-4724-a60e-f55e2cb6c932-multus-daemon-config\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.087418 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.087329 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-etc-kubernetes\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.087418 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.087373 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5d9r\" (UniqueName: \"kubernetes.io/projected/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-kube-api-access-s5d9r\") pod \"network-metrics-daemon-vz9g7\" (UID: \"c85b7d1f-4d4b-4fdd-9427-d1c25942aebe\") " pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:08:54.087538 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.087417 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-etc-kubernetes\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.087538 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.087465 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/80ab5903-b44d-4517-8c0e-fe4abf44d298-device-dir\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:54.087632 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.087591 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/80ab5903-b44d-4517-8c0e-fe4abf44d298-device-dir\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:54.087632 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.087592 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-modprobe-d\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.089650 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.089420 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-sys\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.089650 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.089483 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-multus-cni-dir\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.089650 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.089532 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs\") pod \"network-metrics-daemon-vz9g7\" (UID: \"c85b7d1f-4d4b-4fdd-9427-d1c25942aebe\") " pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:08:54.089650 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.089563 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dknrf\" (UniqueName: \"kubernetes.io/projected/49cf62c9-8b96-43a4-afa7-ac210e06fb8e-kube-api-access-dknrf\") pod \"iptables-alerter-ljsf5\" (UID: \"49cf62c9-8b96-43a4-afa7-ac210e06fb8e\") " pod="openshift-network-operator/iptables-alerter-ljsf5" Jun 15 03:08:54.089650 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.089589 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d75956ab-4288-4724-a60e-f55e2cb6c932-cni-binary-copy\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.089650 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.089617 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-host-run-netns\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.089998 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.089638 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-multus-cni-dir\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.089998 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.089676 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-multus-conf-dir\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.089998 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.089718 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ff9255c7-dc08-4be9-9087-b749284c6358-hosts-file\") pod \"node-resolver-lj57j\" (UID: \"ff9255c7-dc08-4be9-9087-b749284c6358\") " pod="openshift-dns/node-resolver-lj57j" Jun 15 03:08:54.089998 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.089734 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-multus-conf-dir\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.089998 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.089803 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7fe3e8d4-b5da-4511-a7f8-f28596c5f336-serviceca\") pod \"node-ca-xhccz\" (UID: \"7fe3e8d4-b5da-4511-a7f8-f28596c5f336\") " pod="openshift-image-registry/node-ca-xhccz" Jun 15 03:08:54.089998 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.089819 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-host-run-netns\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.089998 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.089860 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ff9255c7-dc08-4be9-9087-b749284c6358-hosts-file\") pod \"node-resolver-lj57j\" (UID: \"ff9255c7-dc08-4be9-9087-b749284c6358\") " pod="openshift-dns/node-resolver-lj57j" Jun 15 03:08:54.089998 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.089870 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3db17e5-7acd-462b-a6f2-b11617619628-kubelet-config\") pod \"global-pull-secret-syncer-mbtvm\" (UID: \"a3db17e5-7acd-462b-a6f2-b11617619628\") " pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:08:54.089998 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:54.089907 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:08:54.089998 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.089911 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-kubernetes\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.090400 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.090152 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d75956ab-4288-4724-a60e-f55e2cb6c932-cni-binary-copy\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.091049 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:54.090678 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs podName:c85b7d1f-4d4b-4fdd-9427-d1c25942aebe nodeName:}" failed. No retries permitted until 2026-06-15 03:08:54.589987978 +0000 UTC m=+2.901126973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs") pod "network-metrics-daemon-vz9g7" (UID: "c85b7d1f-4d4b-4fdd-9427-d1c25942aebe") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:08:54.091049 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.090752 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-cni-netd\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.091049 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.090953 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/795cf261-afbb-4f3f-84a1-617bc13d25f9-tmp\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.091049 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.090985 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-host-run-multus-certs\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.091049 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091014 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-549n9\" (UniqueName: \"kubernetes.io/projected/ff9255c7-dc08-4be9-9087-b749284c6358-kube-api-access-549n9\") pod \"node-resolver-lj57j\" (UID: \"ff9255c7-dc08-4be9-9087-b749284c6358\") " pod="openshift-dns/node-resolver-lj57j" Jun 15 03:08:54.091049 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091043 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3db17e5-7acd-462b-a6f2-b11617619628-dbus\") pod \"global-pull-secret-syncer-mbtvm\" (UID: \"a3db17e5-7acd-462b-a6f2-b11617619628\") " pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:08:54.091409 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091074 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/28f12465-f135-4586-a19f-98430d1f26b0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.091409 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091095 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-host-run-multus-certs\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.091409 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091101 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/49cf62c9-8b96-43a4-afa7-ac210e06fb8e-host-slash\") pod \"iptables-alerter-ljsf5\" (UID: \"49cf62c9-8b96-43a4-afa7-ac210e06fb8e\") " pod="openshift-network-operator/iptables-alerter-ljsf5" Jun 15 03:08:54.091409 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091147 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-host-run-k8s-cni-cncf-io\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.091409 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091192 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-hostroot\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.091409 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091221 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/789deb33-3a27-4d7f-881f-31f889f46e0e-ovnkube-script-lib\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.091409 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091285 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-host-run-k8s-cni-cncf-io\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.091409 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091283 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxx6v\" (UniqueName: \"kubernetes.io/projected/789deb33-3a27-4d7f-881f-31f889f46e0e-kube-api-access-qxx6v\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.091409 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091338 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-hostroot\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.091409 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091344 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-system-cni-dir\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.091409 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091408 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-host-var-lib-kubelet\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.091805 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091427 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-system-cni-dir\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.091805 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091442 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5khg7\" (UniqueName: \"kubernetes.io/projected/d75956ab-4288-4724-a60e-f55e2cb6c932-kube-api-access-5khg7\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.091805 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091466 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/80ab5903-b44d-4517-8c0e-fe4abf44d298-etc-selinux\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:54.091805 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091486 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/28f12465-f135-4586-a19f-98430d1f26b0-cnibin\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.091805 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091431 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-host-var-lib-kubelet\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.091805 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091511 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/28f12465-f135-4586-a19f-98430d1f26b0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.091805 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091634 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/80ab5903-b44d-4517-8c0e-fe4abf44d298-etc-selinux\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:54.092073 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.092052 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-multus-socket-dir-parent\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.092129 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.091629 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-multus-socket-dir-parent\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.092177 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.092109 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/80ab5903-b44d-4517-8c0e-fe4abf44d298-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:54.092177 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.092159 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-run\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.092259 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.092183 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/28f12465-f135-4586-a19f-98430d1f26b0-system-cni-dir\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.092259 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.092214 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/789deb33-3a27-4d7f-881f-31f889f46e0e-ovnkube-config\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.092259 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.092239 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-host-var-lib-cni-bin\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.092259 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.092247 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/80ab5903-b44d-4517-8c0e-fe4abf44d298-kubelet-dir\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:54.092259 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.092255 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nk2w\" (UniqueName: \"kubernetes.io/projected/28f12465-f135-4586-a19f-98430d1f26b0-kube-api-access-2nk2w\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.092433 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.092275 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-run-systemd\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.092433 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.092299 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/80ab5903-b44d-4517-8c0e-fe4abf44d298-sys-fs\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:54.092433 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.092300 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d75956ab-4288-4724-a60e-f55e2cb6c932-host-var-lib-cni-bin\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.092433 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.092319 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-host\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.092433 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.092343 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/80ab5903-b44d-4517-8c0e-fe4abf44d298-sys-fs\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:54.092433 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.092373 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-run-netns\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.092433 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.092397 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-run-ovn\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.092433 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.092434 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/49cf62c9-8b96-43a4-afa7-ac210e06fb8e-iptables-alerter-script\") pod \"iptables-alerter-ljsf5\" (UID: \"49cf62c9-8b96-43a4-afa7-ac210e06fb8e\") " pod="openshift-network-operator/iptables-alerter-ljsf5" Jun 15 03:08:54.092729 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.092460 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff9255c7-dc08-4be9-9087-b749284c6358-tmp-dir\") pod \"node-resolver-lj57j\" (UID: \"ff9255c7-dc08-4be9-9087-b749284c6358\") " pod="openshift-dns/node-resolver-lj57j" Jun 15 03:08:54.092729 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.092481 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7fe3e8d4-b5da-4511-a7f8-f28596c5f336-host\") pod \"node-ca-xhccz\" (UID: \"7fe3e8d4-b5da-4511-a7f8-f28596c5f336\") " pod="openshift-image-registry/node-ca-xhccz" Jun 15 03:08:54.092729 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.092679 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff9255c7-dc08-4be9-9087-b749284c6358-tmp-dir\") pod \"node-resolver-lj57j\" (UID: \"ff9255c7-dc08-4be9-9087-b749284c6358\") " pod="openshift-dns/node-resolver-lj57j" Jun 15 03:08:54.097540 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.097521 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 15 03:08:54.100955 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.100905 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4695v\" (UniqueName: \"kubernetes.io/projected/80ab5903-b44d-4517-8c0e-fe4abf44d298-kube-api-access-4695v\") pod \"aws-ebs-csi-driver-node-cfz4z\" (UID: \"80ab5903-b44d-4517-8c0e-fe4abf44d298\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:54.103178 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.103157 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5d9r\" (UniqueName: \"kubernetes.io/projected/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-kube-api-access-s5d9r\") pod \"network-metrics-daemon-vz9g7\" (UID: \"c85b7d1f-4d4b-4fdd-9427-d1c25942aebe\") " pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:08:54.109942 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.109921 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-549n9\" (UniqueName: \"kubernetes.io/projected/ff9255c7-dc08-4be9-9087-b749284c6358-kube-api-access-549n9\") pod \"node-resolver-lj57j\" (UID: \"ff9255c7-dc08-4be9-9087-b749284c6358\") " pod="openshift-dns/node-resolver-lj57j" Jun 15 03:08:54.110115 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.110096 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5khg7\" (UniqueName: \"kubernetes.io/projected/d75956ab-4288-4724-a60e-f55e2cb6c932-kube-api-access-5khg7\") pod \"multus-tpvc8\" (UID: \"d75956ab-4288-4724-a60e-f55e2cb6c932\") " pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.193253 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193221 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-host\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.193253 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193249 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-run-netns\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.193483 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193267 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-run-ovn\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.193483 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193333 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-host\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.193483 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193335 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-run-netns\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.193483 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193361 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/49cf62c9-8b96-43a4-afa7-ac210e06fb8e-iptables-alerter-script\") pod \"iptables-alerter-ljsf5\" (UID: \"49cf62c9-8b96-43a4-afa7-ac210e06fb8e\") " pod="openshift-network-operator/iptables-alerter-ljsf5" Jun 15 03:08:54.193483 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193369 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-run-ovn\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.193483 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193380 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7fe3e8d4-b5da-4511-a7f8-f28596c5f336-host\") pod \"node-ca-xhccz\" (UID: \"7fe3e8d4-b5da-4511-a7f8-f28596c5f336\") " pod="openshift-image-registry/node-ca-xhccz" Jun 15 03:08:54.193483 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193396 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0a161380-ecd7-4473-9cc2-e3041ad5571d-konnectivity-ca\") pod \"konnectivity-agent-s4gh2\" (UID: \"0a161380-ecd7-4473-9cc2-e3041ad5571d\") " pod="kube-system/konnectivity-agent-s4gh2" Jun 15 03:08:54.193483 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193410 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-systemd\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.193483 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193421 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7fe3e8d4-b5da-4511-a7f8-f28596c5f336-host\") pod \"node-ca-xhccz\" (UID: \"7fe3e8d4-b5da-4511-a7f8-f28596c5f336\") " pod="openshift-image-registry/node-ca-xhccz" Jun 15 03:08:54.193483 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193426 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-var-lib-kubelet\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.193483 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193450 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-kubelet\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.193483 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193465 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-systemd-units\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.193483 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193479 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-var-lib-kubelet\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.193483 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193482 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjjrr\" (UniqueName: \"kubernetes.io/projected/795cf261-afbb-4f3f-84a1-617bc13d25f9-kube-api-access-gjjrr\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.194207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193489 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-systemd\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.194207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193528 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-kubelet\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.194207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193549 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr2g5\" (UniqueName: \"kubernetes.io/projected/7fe3e8d4-b5da-4511-a7f8-f28596c5f336-kube-api-access-tr2g5\") pod \"node-ca-xhccz\" (UID: \"7fe3e8d4-b5da-4511-a7f8-f28596c5f336\") " pod="openshift-image-registry/node-ca-xhccz" Jun 15 03:08:54.194207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193575 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-systemd-units\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.194207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193580 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-sysctl-d\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.194207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193620 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-tuned\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.194207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193646 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/28f12465-f135-4586-a19f-98430d1f26b0-os-release\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.194207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193674 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/28f12465-f135-4586-a19f-98430d1f26b0-cni-binary-copy\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.194207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193699 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/28f12465-f135-4586-a19f-98430d1f26b0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.194207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193705 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-sysctl-d\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.194207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193726 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-run-ovn-kubernetes\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.194207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193746 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/28f12465-f135-4586-a19f-98430d1f26b0-os-release\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.194207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193753 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-sysconfig\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.194207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193783 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-etc-openvswitch\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.194207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193797 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-sysconfig\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.194207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193810 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.194207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193851 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-run-ovn-kubernetes\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.195035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193857 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/789deb33-3a27-4d7f-881f-31f889f46e0e-ovn-node-metrics-cert\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.195035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193885 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret\") pod \"global-pull-secret-syncer-mbtvm\" (UID: \"a3db17e5-7acd-462b-a6f2-b11617619628\") " pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:08:54.195035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193889 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-etc-openvswitch\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.195035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193911 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-log-socket\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.195035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193933 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-cni-bin\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.195035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193957 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5pnn\" (UniqueName: \"kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn\") pod \"network-check-target-nhnmr\" (UID: \"5488015e-fad7-4f27-b7a0-62d9bb3d7da0\") " pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:08:54.195035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193977 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/49cf62c9-8b96-43a4-afa7-ac210e06fb8e-iptables-alerter-script\") pod \"iptables-alerter-ljsf5\" (UID: \"49cf62c9-8b96-43a4-afa7-ac210e06fb8e\") " pod="openshift-network-operator/iptables-alerter-ljsf5" Jun 15 03:08:54.195035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193985 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0a161380-ecd7-4473-9cc2-e3041ad5571d-agent-certs\") pod \"konnectivity-agent-s4gh2\" (UID: \"0a161380-ecd7-4473-9cc2-e3041ad5571d\") " pod="kube-system/konnectivity-agent-s4gh2" Jun 15 03:08:54.195035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194010 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-sysctl-conf\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.195035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194035 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/789deb33-3a27-4d7f-881f-31f889f46e0e-env-overrides\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.195035 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:54.194056 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 03:08:54.195035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194290 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/28f12465-f135-4586-a19f-98430d1f26b0-cni-binary-copy\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.195035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194061 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-node-log\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.195035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194105 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-node-log\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.195035 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:54.194327 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret podName:a3db17e5-7acd-462b-a6f2-b11617619628 nodeName:}" failed. No retries permitted until 2026-06-15 03:08:54.694308782 +0000 UTC m=+3.005447754 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret") pod "global-pull-secret-syncer-mbtvm" (UID: "a3db17e5-7acd-462b-a6f2-b11617619628") : object "kube-system"/"original-pull-secret" not registered Jun 15 03:08:54.195035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194225 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.195035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194307 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-sysctl-conf\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.195901 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194111 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0a161380-ecd7-4473-9cc2-e3041ad5571d-konnectivity-ca\") pod \"konnectivity-agent-s4gh2\" (UID: \"0a161380-ecd7-4473-9cc2-e3041ad5571d\") " pod="kube-system/konnectivity-agent-s4gh2" Jun 15 03:08:54.195901 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.193984 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-log-socket\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.195901 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194441 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-lib-modules\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.195901 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194490 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-slash\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.195901 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194519 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-var-lib-openvswitch\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.195901 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194545 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-run-openvswitch\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.195901 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194569 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-slash\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.195901 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194331 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/28f12465-f135-4586-a19f-98430d1f26b0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.195901 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194571 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-lib-modules\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.195901 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194590 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-modprobe-d\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.195901 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194057 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-cni-bin\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.195901 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194621 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-var-lib-openvswitch\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.195901 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194625 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-sys\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.195901 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194660 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-run-openvswitch\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.195901 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194670 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dknrf\" (UniqueName: \"kubernetes.io/projected/49cf62c9-8b96-43a4-afa7-ac210e06fb8e-kube-api-access-dknrf\") pod \"iptables-alerter-ljsf5\" (UID: \"49cf62c9-8b96-43a4-afa7-ac210e06fb8e\") " pod="openshift-network-operator/iptables-alerter-ljsf5" Jun 15 03:08:54.195901 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194687 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-modprobe-d\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.195901 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194710 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7fe3e8d4-b5da-4511-a7f8-f28596c5f336-serviceca\") pod \"node-ca-xhccz\" (UID: \"7fe3e8d4-b5da-4511-a7f8-f28596c5f336\") " pod="openshift-image-registry/node-ca-xhccz" Jun 15 03:08:54.196701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194737 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3db17e5-7acd-462b-a6f2-b11617619628-kubelet-config\") pod \"global-pull-secret-syncer-mbtvm\" (UID: \"a3db17e5-7acd-462b-a6f2-b11617619628\") " pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:08:54.196701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194760 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-kubernetes\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.196701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194711 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-sys\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.196701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194791 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-cni-netd\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.196701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194815 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/795cf261-afbb-4f3f-84a1-617bc13d25f9-tmp\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.196701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194859 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3db17e5-7acd-462b-a6f2-b11617619628-dbus\") pod \"global-pull-secret-syncer-mbtvm\" (UID: \"a3db17e5-7acd-462b-a6f2-b11617619628\") " pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:08:54.196701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194885 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/28f12465-f135-4586-a19f-98430d1f26b0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.196701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194908 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/789deb33-3a27-4d7f-881f-31f889f46e0e-env-overrides\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.196701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194917 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/49cf62c9-8b96-43a4-afa7-ac210e06fb8e-host-slash\") pod \"iptables-alerter-ljsf5\" (UID: \"49cf62c9-8b96-43a4-afa7-ac210e06fb8e\") " pod="openshift-network-operator/iptables-alerter-ljsf5" Jun 15 03:08:54.196701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194949 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/789deb33-3a27-4d7f-881f-31f889f46e0e-ovnkube-script-lib\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.196701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194952 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-host-cni-netd\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.196701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.194971 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxx6v\" (UniqueName: \"kubernetes.io/projected/789deb33-3a27-4d7f-881f-31f889f46e0e-kube-api-access-qxx6v\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.196701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.195021 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/28f12465-f135-4586-a19f-98430d1f26b0-cnibin\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.196701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.195048 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/28f12465-f135-4586-a19f-98430d1f26b0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.196701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.195076 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-run\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.196701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.195100 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/28f12465-f135-4586-a19f-98430d1f26b0-system-cni-dir\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.196701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.195126 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/789deb33-3a27-4d7f-881f-31f889f46e0e-ovnkube-config\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.197526 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.195154 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nk2w\" (UniqueName: \"kubernetes.io/projected/28f12465-f135-4586-a19f-98430d1f26b0-kube-api-access-2nk2w\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.197526 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.195183 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-run-systemd\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.197526 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.195187 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7fe3e8d4-b5da-4511-a7f8-f28596c5f336-serviceca\") pod \"node-ca-xhccz\" (UID: \"7fe3e8d4-b5da-4511-a7f8-f28596c5f336\") " pod="openshift-image-registry/node-ca-xhccz" Jun 15 03:08:54.197526 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.195211 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/49cf62c9-8b96-43a4-afa7-ac210e06fb8e-host-slash\") pod \"iptables-alerter-ljsf5\" (UID: \"49cf62c9-8b96-43a4-afa7-ac210e06fb8e\") " pod="openshift-network-operator/iptables-alerter-ljsf5" Jun 15 03:08:54.197526 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.195024 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3db17e5-7acd-462b-a6f2-b11617619628-dbus\") pod \"global-pull-secret-syncer-mbtvm\" (UID: \"a3db17e5-7acd-462b-a6f2-b11617619628\") " pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:08:54.197526 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.195268 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/28f12465-f135-4586-a19f-98430d1f26b0-cnibin\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.197526 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.195271 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-run\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.197526 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.195323 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3db17e5-7acd-462b-a6f2-b11617619628-kubelet-config\") pod \"global-pull-secret-syncer-mbtvm\" (UID: \"a3db17e5-7acd-462b-a6f2-b11617619628\") " pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:08:54.197526 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.195365 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-kubernetes\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.197526 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.195399 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/28f12465-f135-4586-a19f-98430d1f26b0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.197526 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.195401 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/789deb33-3a27-4d7f-881f-31f889f46e0e-run-systemd\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.197526 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.195438 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/28f12465-f135-4586-a19f-98430d1f26b0-system-cni-dir\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.197526 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.195653 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/789deb33-3a27-4d7f-881f-31f889f46e0e-ovnkube-script-lib\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.197526 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.195729 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/28f12465-f135-4586-a19f-98430d1f26b0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.197526 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.195947 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/789deb33-3a27-4d7f-881f-31f889f46e0e-ovnkube-config\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.197526 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.197098 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/795cf261-afbb-4f3f-84a1-617bc13d25f9-tmp\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.197526 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.197201 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/789deb33-3a27-4d7f-881f-31f889f46e0e-ovn-node-metrics-cert\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.198015 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.197250 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0a161380-ecd7-4473-9cc2-e3041ad5571d-agent-certs\") pod \"konnectivity-agent-s4gh2\" (UID: \"0a161380-ecd7-4473-9cc2-e3041ad5571d\") " pod="kube-system/konnectivity-agent-s4gh2" Jun 15 03:08:54.198015 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.197593 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/795cf261-afbb-4f3f-84a1-617bc13d25f9-etc-tuned\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.201301 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.201280 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjjrr\" (UniqueName: \"kubernetes.io/projected/795cf261-afbb-4f3f-84a1-617bc13d25f9-kube-api-access-gjjrr\") pod \"tuned-t98lm\" (UID: \"795cf261-afbb-4f3f-84a1-617bc13d25f9\") " pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.206252 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:54.206205 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 03:08:54.206252 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:54.206231 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 03:08:54.206252 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:54.206246 2562 projected.go:196] Error preparing data for projected volume kube-api-access-t5pnn for pod openshift-network-diagnostics/network-check-target-nhnmr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:08:54.206405 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:54.206331 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn podName:5488015e-fad7-4f27-b7a0-62d9bb3d7da0 nodeName:}" failed. No retries permitted until 2026-06-15 03:08:54.706312263 +0000 UTC m=+3.017451234 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-t5pnn" (UniqueName: "kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn") pod "network-check-target-nhnmr" (UID: "5488015e-fad7-4f27-b7a0-62d9bb3d7da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:08:54.208010 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.207981 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr2g5\" (UniqueName: \"kubernetes.io/projected/7fe3e8d4-b5da-4511-a7f8-f28596c5f336-kube-api-access-tr2g5\") pod \"node-ca-xhccz\" (UID: \"7fe3e8d4-b5da-4511-a7f8-f28596c5f336\") " pod="openshift-image-registry/node-ca-xhccz" Jun 15 03:08:54.214533 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.214509 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dknrf\" (UniqueName: \"kubernetes.io/projected/49cf62c9-8b96-43a4-afa7-ac210e06fb8e-kube-api-access-dknrf\") pod \"iptables-alerter-ljsf5\" (UID: \"49cf62c9-8b96-43a4-afa7-ac210e06fb8e\") " pod="openshift-network-operator/iptables-alerter-ljsf5" Jun 15 03:08:54.214623 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.214596 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nk2w\" (UniqueName: \"kubernetes.io/projected/28f12465-f135-4586-a19f-98430d1f26b0-kube-api-access-2nk2w\") pod \"multus-additional-cni-plugins-pn582\" (UID: \"28f12465-f135-4586-a19f-98430d1f26b0\") " pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.216569 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.216547 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxx6v\" (UniqueName: \"kubernetes.io/projected/789deb33-3a27-4d7f-881f-31f889f46e0e-kube-api-access-qxx6v\") pod \"ovnkube-node-9r9mn\" (UID: \"789deb33-3a27-4d7f-881f-31f889f46e0e\") " pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.278682 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.278650 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tpvc8" Jun 15 03:08:54.287153 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.287124 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 15 03:08:54.290822 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.290796 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" Jun 15 03:08:54.299613 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.299594 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lj57j" Jun 15 03:08:54.306259 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.306243 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xhccz" Jun 15 03:08:54.313860 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.313842 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pn582" Jun 15 03:08:54.320498 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.320478 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:08:54.327150 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.327132 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s4gh2" Jun 15 03:08:54.332737 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.332720 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-t98lm" Jun 15 03:08:54.340224 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.340208 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ljsf5" Jun 15 03:08:54.597975 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.597889 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs\") pod \"network-metrics-daemon-vz9g7\" (UID: \"c85b7d1f-4d4b-4fdd-9427-d1c25942aebe\") " pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:08:54.598111 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:54.598038 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:08:54.598111 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:54.598105 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs podName:c85b7d1f-4d4b-4fdd-9427-d1c25942aebe nodeName:}" failed. No retries permitted until 2026-06-15 03:08:55.598089553 +0000 UTC m=+3.909228520 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs") pod "network-metrics-daemon-vz9g7" (UID: "c85b7d1f-4d4b-4fdd-9427-d1c25942aebe") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:08:54.699141 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.699108 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret\") pod \"global-pull-secret-syncer-mbtvm\" (UID: \"a3db17e5-7acd-462b-a6f2-b11617619628\") " pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:08:54.699294 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:54.699215 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 03:08:54.699294 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:54.699271 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret podName:a3db17e5-7acd-462b-a6f2-b11617619628 nodeName:}" failed. No retries permitted until 2026-06-15 03:08:55.699254388 +0000 UTC m=+4.010393362 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret") pod "global-pull-secret-syncer-mbtvm" (UID: "a3db17e5-7acd-462b-a6f2-b11617619628") : object "kube-system"/"original-pull-secret" not registered Jun 15 03:08:54.800064 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:54.800026 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5pnn\" (UniqueName: \"kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn\") pod \"network-check-target-nhnmr\" (UID: \"5488015e-fad7-4f27-b7a0-62d9bb3d7da0\") " pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:08:54.800251 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:54.800213 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 03:08:54.800251 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:54.800238 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 03:08:54.800251 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:54.800251 2562 projected.go:196] Error preparing data for projected volume kube-api-access-t5pnn for pod openshift-network-diagnostics/network-check-target-nhnmr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:08:54.800402 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:54.800317 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn podName:5488015e-fad7-4f27-b7a0-62d9bb3d7da0 nodeName:}" failed. No retries permitted until 2026-06-15 03:08:55.800298502 +0000 UTC m=+4.111437472 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-t5pnn" (UniqueName: "kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn") pod "network-check-target-nhnmr" (UID: "5488015e-fad7-4f27-b7a0-62d9bb3d7da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:08:54.869800 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:54.869764 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff9255c7_dc08_4be9_9087_b749284c6358.slice/crio-c558a77577b6920b2e5c9f79042606a0486f109cf418645db7fa1fc79f898370 WatchSource:0}: Error finding container c558a77577b6920b2e5c9f79042606a0486f109cf418645db7fa1fc79f898370: Status 404 returned error can't find the container with id c558a77577b6920b2e5c9f79042606a0486f109cf418645db7fa1fc79f898370 Jun 15 03:08:54.871404 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:54.871377 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod795cf261_afbb_4f3f_84a1_617bc13d25f9.slice/crio-7da60d006f47d53c0c1161747877e40d8db7df12131ffbaf6d2e4945fc18cfd3 WatchSource:0}: Error finding container 7da60d006f47d53c0c1161747877e40d8db7df12131ffbaf6d2e4945fc18cfd3: Status 404 returned error can't find the container with id 7da60d006f47d53c0c1161747877e40d8db7df12131ffbaf6d2e4945fc18cfd3 Jun 15 03:08:54.872608 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:54.872580 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd75956ab_4288_4724_a60e_f55e2cb6c932.slice/crio-2cdc0b6107269979f48c16bfa8281880d805dea7ab213d919039f63673a5db81 WatchSource:0}: Error finding container 2cdc0b6107269979f48c16bfa8281880d805dea7ab213d919039f63673a5db81: Status 404 returned error can't find the container with id 2cdc0b6107269979f48c16bfa8281880d805dea7ab213d919039f63673a5db81 Jun 15 03:08:54.876800 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:54.876738 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a161380_ecd7_4473_9cc2_e3041ad5571d.slice/crio-740a11fda715a1216922f803255ef8b5bc3e3de417182fdad12cf71b173332da WatchSource:0}: Error finding container 740a11fda715a1216922f803255ef8b5bc3e3de417182fdad12cf71b173332da: Status 404 returned error can't find the container with id 740a11fda715a1216922f803255ef8b5bc3e3de417182fdad12cf71b173332da Jun 15 03:08:54.877301 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:54.877276 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28f12465_f135_4586_a19f_98430d1f26b0.slice/crio-b14b3bfb0d88bb0c5259ac9e36165ef6e8ff3b132e78098a98f1c2acbe4d94ff WatchSource:0}: Error finding container b14b3bfb0d88bb0c5259ac9e36165ef6e8ff3b132e78098a98f1c2acbe4d94ff: Status 404 returned error can't find the container with id b14b3bfb0d88bb0c5259ac9e36165ef6e8ff3b132e78098a98f1c2acbe4d94ff Jun 15 03:08:54.899131 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:54.899070 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod789deb33_3a27_4d7f_881f_31f889f46e0e.slice/crio-686229b3f10a9919a0bc289f84a91ed961039274ad701ba46b61aa708c82ae55 WatchSource:0}: Error finding container 686229b3f10a9919a0bc289f84a91ed961039274ad701ba46b61aa708c82ae55: Status 404 returned error can't find the container with id 686229b3f10a9919a0bc289f84a91ed961039274ad701ba46b61aa708c82ae55 Jun 15 03:08:54.900087 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:54.900057 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80ab5903_b44d_4517_8c0e_fe4abf44d298.slice/crio-07c20733bebf2c8a425565e54950d305763b0243c2c97fa53f9f09bc40eb3abd WatchSource:0}: Error finding container 07c20733bebf2c8a425565e54950d305763b0243c2c97fa53f9f09bc40eb3abd: Status 404 returned error can't find the container with id 07c20733bebf2c8a425565e54950d305763b0243c2c97fa53f9f09bc40eb3abd Jun 15 03:08:54.900723 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:08:54.900704 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fe3e8d4_b5da_4511_a7f8_f28596c5f336.slice/crio-e69a61437a0b2236b9b31b288c6bf93783cf98e22621f756822e1c84d2a7351d WatchSource:0}: Error finding container e69a61437a0b2236b9b31b288c6bf93783cf98e22621f756822e1c84d2a7351d: Status 404 returned error can't find the container with id e69a61437a0b2236b9b31b288c6bf93783cf98e22621f756822e1c84d2a7351d Jun 15 03:08:55.026818 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:55.026615 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-14 03:03:54 +0000 UTC" deadline="2028-03-16 15:24:12.623930906 +0000 UTC" Jun 15 03:08:55.026818 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:55.026815 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15372h15m17.597120005s" Jun 15 03:08:55.047667 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:55.047635 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xhccz" event={"ID":"7fe3e8d4-b5da-4511-a7f8-f28596c5f336","Type":"ContainerStarted","Data":"e69a61437a0b2236b9b31b288c6bf93783cf98e22621f756822e1c84d2a7351d"} Jun 15 03:08:55.048906 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:55.048872 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" event={"ID":"789deb33-3a27-4d7f-881f-31f889f46e0e","Type":"ContainerStarted","Data":"686229b3f10a9919a0bc289f84a91ed961039274ad701ba46b61aa708c82ae55"} Jun 15 03:08:55.049996 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:55.049972 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ljsf5" event={"ID":"49cf62c9-8b96-43a4-afa7-ac210e06fb8e","Type":"ContainerStarted","Data":"670a92e02dfb84cbc05c4177d4313a96e268371401231ce5850af8320f9638c3"} Jun 15 03:08:55.051142 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:55.051119 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s4gh2" event={"ID":"0a161380-ecd7-4473-9cc2-e3041ad5571d","Type":"ContainerStarted","Data":"740a11fda715a1216922f803255ef8b5bc3e3de417182fdad12cf71b173332da"} Jun 15 03:08:55.052147 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:55.052123 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tpvc8" event={"ID":"d75956ab-4288-4724-a60e-f55e2cb6c932","Type":"ContainerStarted","Data":"2cdc0b6107269979f48c16bfa8281880d805dea7ab213d919039f63673a5db81"} Jun 15 03:08:55.053165 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:55.053146 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lj57j" event={"ID":"ff9255c7-dc08-4be9-9087-b749284c6358","Type":"ContainerStarted","Data":"c558a77577b6920b2e5c9f79042606a0486f109cf418645db7fa1fc79f898370"} Jun 15 03:08:55.054198 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:55.054180 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" event={"ID":"80ab5903-b44d-4517-8c0e-fe4abf44d298","Type":"ContainerStarted","Data":"07c20733bebf2c8a425565e54950d305763b0243c2c97fa53f9f09bc40eb3abd"} Jun 15 03:08:55.055236 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:55.055214 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn582" event={"ID":"28f12465-f135-4586-a19f-98430d1f26b0","Type":"ContainerStarted","Data":"b14b3bfb0d88bb0c5259ac9e36165ef6e8ff3b132e78098a98f1c2acbe4d94ff"} Jun 15 03:08:55.056274 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:55.056254 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t98lm" event={"ID":"795cf261-afbb-4f3f-84a1-617bc13d25f9","Type":"ContainerStarted","Data":"7da60d006f47d53c0c1161747877e40d8db7df12131ffbaf6d2e4945fc18cfd3"} Jun 15 03:08:55.058109 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:55.058089 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal" event={"ID":"a43d74820b080a54928ef60318b8a3ec","Type":"ContainerStarted","Data":"e9c0077470c9451479440736e68941d4f9944587feee8ab3de2f64f665213362"} Jun 15 03:08:55.608054 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:55.607963 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs\") pod \"network-metrics-daemon-vz9g7\" (UID: \"c85b7d1f-4d4b-4fdd-9427-d1c25942aebe\") " pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:08:55.608271 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:55.608097 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:08:55.608271 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:55.608154 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs podName:c85b7d1f-4d4b-4fdd-9427-d1c25942aebe nodeName:}" failed. No retries permitted until 2026-06-15 03:08:57.608134963 +0000 UTC m=+5.919273935 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs") pod "network-metrics-daemon-vz9g7" (UID: "c85b7d1f-4d4b-4fdd-9427-d1c25942aebe") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:08:55.708485 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:55.708446 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret\") pod \"global-pull-secret-syncer-mbtvm\" (UID: \"a3db17e5-7acd-462b-a6f2-b11617619628\") " pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:08:55.708696 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:55.708658 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 03:08:55.708895 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:55.708720 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret podName:a3db17e5-7acd-462b-a6f2-b11617619628 nodeName:}" failed. No retries permitted until 2026-06-15 03:08:57.708701789 +0000 UTC m=+6.019840759 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret") pod "global-pull-secret-syncer-mbtvm" (UID: "a3db17e5-7acd-462b-a6f2-b11617619628") : object "kube-system"/"original-pull-secret" not registered Jun 15 03:08:55.809777 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:55.809740 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5pnn\" (UniqueName: \"kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn\") pod \"network-check-target-nhnmr\" (UID: \"5488015e-fad7-4f27-b7a0-62d9bb3d7da0\") " pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:08:55.809977 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:55.809947 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 03:08:55.809977 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:55.809967 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 03:08:55.810078 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:55.809981 2562 projected.go:196] Error preparing data for projected volume kube-api-access-t5pnn for pod openshift-network-diagnostics/network-check-target-nhnmr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:08:55.810078 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:55.810043 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn podName:5488015e-fad7-4f27-b7a0-62d9bb3d7da0 nodeName:}" failed. No retries permitted until 2026-06-15 03:08:57.810022097 +0000 UTC m=+6.121161082 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-t5pnn" (UniqueName: "kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn") pod "network-check-target-nhnmr" (UID: "5488015e-fad7-4f27-b7a0-62d9bb3d7da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:08:56.027347 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:56.027272 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-14 03:03:54 +0000 UTC" deadline="2028-02-01 12:56:32.69484276 +0000 UTC" Jun 15 03:08:56.027347 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:56.027308 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14313h47m36.667539132s" Jun 15 03:08:56.042939 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:56.042184 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:08:56.042939 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:56.042313 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vz9g7" podUID="c85b7d1f-4d4b-4fdd-9427-d1c25942aebe" Jun 15 03:08:56.042939 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:56.042678 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:08:56.042939 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:56.042782 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nhnmr" podUID="5488015e-fad7-4f27-b7a0-62d9bb3d7da0" Jun 15 03:08:56.043628 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:56.043446 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:08:56.043628 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:56.043554 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mbtvm" podUID="a3db17e5-7acd-462b-a6f2-b11617619628" Jun 15 03:08:56.078591 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:56.078558 2562 generic.go:356] "Generic (PLEG): container finished" podID="1cd3eb1948b1e36499470a2dffb1c745" containerID="cecfa5149b0ccdd596d7768a4372fc3d97cde9e796a4852d47d5fd53455a4d02" exitCode=0 Jun 15 03:08:56.078766 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:56.078703 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" event={"ID":"1cd3eb1948b1e36499470a2dffb1c745","Type":"ContainerDied","Data":"cecfa5149b0ccdd596d7768a4372fc3d97cde9e796a4852d47d5fd53455a4d02"} Jun 15 03:08:56.091906 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:56.091850 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal" podStartSLOduration=3.091816287 podStartE2EDuration="3.091816287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:08:55.073446456 +0000 UTC m=+3.384585457" watchObservedRunningTime="2026-06-15 03:08:56.091816287 +0000 UTC m=+4.402955280" Jun 15 03:08:57.110148 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:57.110007 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" event={"ID":"1cd3eb1948b1e36499470a2dffb1c745","Type":"ContainerStarted","Data":"defaf88152d4988809565204432e31bb3450f96dbb9cd1d767fd3e78e40e4c05"} Jun 15 03:08:57.124346 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:57.124295 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" podStartSLOduration=4.124274746 podStartE2EDuration="4.124274746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:08:57.124204441 +0000 UTC m=+5.435343432" watchObservedRunningTime="2026-06-15 03:08:57.124274746 +0000 UTC m=+5.435413736" Jun 15 03:08:57.635557 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:57.635520 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs\") pod \"network-metrics-daemon-vz9g7\" (UID: \"c85b7d1f-4d4b-4fdd-9427-d1c25942aebe\") " pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:08:57.635771 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:57.635677 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:08:57.635771 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:57.635736 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs podName:c85b7d1f-4d4b-4fdd-9427-d1c25942aebe nodeName:}" failed. No retries permitted until 2026-06-15 03:09:01.635717466 +0000 UTC m=+9.946856440 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs") pod "network-metrics-daemon-vz9g7" (UID: "c85b7d1f-4d4b-4fdd-9427-d1c25942aebe") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:08:57.736745 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:57.736711 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret\") pod \"global-pull-secret-syncer-mbtvm\" (UID: \"a3db17e5-7acd-462b-a6f2-b11617619628\") " pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:08:57.736927 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:57.736899 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 03:08:57.737000 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:57.736961 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret podName:a3db17e5-7acd-462b-a6f2-b11617619628 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:01.736943267 +0000 UTC m=+10.048082252 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret") pod "global-pull-secret-syncer-mbtvm" (UID: "a3db17e5-7acd-462b-a6f2-b11617619628") : object "kube-system"/"original-pull-secret" not registered Jun 15 03:08:57.837509 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:57.837468 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5pnn\" (UniqueName: \"kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn\") pod \"network-check-target-nhnmr\" (UID: \"5488015e-fad7-4f27-b7a0-62d9bb3d7da0\") " pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:08:57.837711 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:57.837692 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 03:08:57.837780 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:57.837721 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 03:08:57.837780 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:57.837735 2562 projected.go:196] Error preparing data for projected volume kube-api-access-t5pnn for pod openshift-network-diagnostics/network-check-target-nhnmr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:08:57.837963 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:57.837795 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn podName:5488015e-fad7-4f27-b7a0-62d9bb3d7da0 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:01.837776142 +0000 UTC m=+10.148915113 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-t5pnn" (UniqueName: "kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn") pod "network-check-target-nhnmr" (UID: "5488015e-fad7-4f27-b7a0-62d9bb3d7da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:08:58.039655 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:58.039575 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:08:58.039808 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:58.039706 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mbtvm" podUID="a3db17e5-7acd-462b-a6f2-b11617619628" Jun 15 03:08:58.040147 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:58.040127 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:08:58.040264 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:58.040244 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vz9g7" podUID="c85b7d1f-4d4b-4fdd-9427-d1c25942aebe" Jun 15 03:08:58.040530 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:08:58.040491 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:08:58.040806 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:08:58.040755 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nhnmr" podUID="5488015e-fad7-4f27-b7a0-62d9bb3d7da0" Jun 15 03:09:00.039511 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:00.039481 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:00.039971 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:00.039526 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:09:00.039971 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:00.039614 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mbtvm" podUID="a3db17e5-7acd-462b-a6f2-b11617619628" Jun 15 03:09:00.039971 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:00.039967 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:00.040134 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:00.040066 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nhnmr" podUID="5488015e-fad7-4f27-b7a0-62d9bb3d7da0" Jun 15 03:09:00.040189 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:00.040160 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vz9g7" podUID="c85b7d1f-4d4b-4fdd-9427-d1c25942aebe" Jun 15 03:09:01.673995 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:01.673778 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs\") pod \"network-metrics-daemon-vz9g7\" (UID: \"c85b7d1f-4d4b-4fdd-9427-d1c25942aebe\") " pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:09:01.673995 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:01.673948 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:09:01.674536 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:01.674016 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs podName:c85b7d1f-4d4b-4fdd-9427-d1c25942aebe nodeName:}" failed. No retries permitted until 2026-06-15 03:09:09.673996036 +0000 UTC m=+17.985135018 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs") pod "network-metrics-daemon-vz9g7" (UID: "c85b7d1f-4d4b-4fdd-9427-d1c25942aebe") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:09:01.774731 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:01.774676 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret\") pod \"global-pull-secret-syncer-mbtvm\" (UID: \"a3db17e5-7acd-462b-a6f2-b11617619628\") " pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:01.774990 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:01.774938 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 03:09:01.775072 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:01.775027 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret podName:a3db17e5-7acd-462b-a6f2-b11617619628 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:09.77500561 +0000 UTC m=+18.086144797 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret") pod "global-pull-secret-syncer-mbtvm" (UID: "a3db17e5-7acd-462b-a6f2-b11617619628") : object "kube-system"/"original-pull-secret" not registered Jun 15 03:09:01.875158 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:01.875119 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5pnn\" (UniqueName: \"kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn\") pod \"network-check-target-nhnmr\" (UID: \"5488015e-fad7-4f27-b7a0-62d9bb3d7da0\") " pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:01.875340 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:01.875265 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 03:09:01.875340 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:01.875285 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 03:09:01.875340 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:01.875298 2562 projected.go:196] Error preparing data for projected volume kube-api-access-t5pnn for pod openshift-network-diagnostics/network-check-target-nhnmr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:09:01.875473 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:01.875360 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn podName:5488015e-fad7-4f27-b7a0-62d9bb3d7da0 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:09.875341519 +0000 UTC m=+18.186480500 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-t5pnn" (UniqueName: "kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn") pod "network-check-target-nhnmr" (UID: "5488015e-fad7-4f27-b7a0-62d9bb3d7da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:09:02.040307 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:02.040221 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:02.040471 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:02.040335 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mbtvm" podUID="a3db17e5-7acd-462b-a6f2-b11617619628" Jun 15 03:09:02.040471 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:02.040381 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:02.040471 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:02.040434 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:09:02.040638 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:02.040531 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vz9g7" podUID="c85b7d1f-4d4b-4fdd-9427-d1c25942aebe" Jun 15 03:09:02.040638 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:02.040595 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nhnmr" podUID="5488015e-fad7-4f27-b7a0-62d9bb3d7da0" Jun 15 03:09:04.039446 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:04.039413 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:04.039446 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:04.039418 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:09:04.039858 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:04.039424 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:04.039858 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:04.039558 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vz9g7" podUID="c85b7d1f-4d4b-4fdd-9427-d1c25942aebe" Jun 15 03:09:04.039858 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:04.039577 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mbtvm" podUID="a3db17e5-7acd-462b-a6f2-b11617619628" Jun 15 03:09:04.039858 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:04.039645 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nhnmr" podUID="5488015e-fad7-4f27-b7a0-62d9bb3d7da0" Jun 15 03:09:06.039568 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:06.039491 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:06.040029 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:06.039492 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:09:06.040029 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:06.039632 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mbtvm" podUID="a3db17e5-7acd-462b-a6f2-b11617619628" Jun 15 03:09:06.040029 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:06.039692 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:06.040029 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:06.039801 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vz9g7" podUID="c85b7d1f-4d4b-4fdd-9427-d1c25942aebe" Jun 15 03:09:06.040029 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:06.039895 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nhnmr" podUID="5488015e-fad7-4f27-b7a0-62d9bb3d7da0" Jun 15 03:09:08.040170 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:08.040141 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:08.040170 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:08.040161 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:09:08.040695 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:08.040142 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:08.040695 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:08.040274 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nhnmr" podUID="5488015e-fad7-4f27-b7a0-62d9bb3d7da0" Jun 15 03:09:08.040695 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:08.040346 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mbtvm" podUID="a3db17e5-7acd-462b-a6f2-b11617619628" Jun 15 03:09:08.040695 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:08.040448 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vz9g7" podUID="c85b7d1f-4d4b-4fdd-9427-d1c25942aebe" Jun 15 03:09:09.730929 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:09.730887 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs\") pod \"network-metrics-daemon-vz9g7\" (UID: \"c85b7d1f-4d4b-4fdd-9427-d1c25942aebe\") " pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:09:09.731398 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:09.731074 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:09:09.731398 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:09.731152 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs podName:c85b7d1f-4d4b-4fdd-9427-d1c25942aebe nodeName:}" failed. No retries permitted until 2026-06-15 03:09:25.73113275 +0000 UTC m=+34.042271732 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs") pod "network-metrics-daemon-vz9g7" (UID: "c85b7d1f-4d4b-4fdd-9427-d1c25942aebe") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:09:09.832385 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:09.832341 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret\") pod \"global-pull-secret-syncer-mbtvm\" (UID: \"a3db17e5-7acd-462b-a6f2-b11617619628\") " pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:09.832552 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:09.832491 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 03:09:09.832622 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:09.832563 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret podName:a3db17e5-7acd-462b-a6f2-b11617619628 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:25.832542124 +0000 UTC m=+34.143681094 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret") pod "global-pull-secret-syncer-mbtvm" (UID: "a3db17e5-7acd-462b-a6f2-b11617619628") : object "kube-system"/"original-pull-secret" not registered Jun 15 03:09:09.933081 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:09.933040 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5pnn\" (UniqueName: \"kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn\") pod \"network-check-target-nhnmr\" (UID: \"5488015e-fad7-4f27-b7a0-62d9bb3d7da0\") " pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:09.933241 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:09.933206 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 03:09:09.933241 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:09.933235 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 03:09:09.933359 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:09.933250 2562 projected.go:196] Error preparing data for projected volume kube-api-access-t5pnn for pod openshift-network-diagnostics/network-check-target-nhnmr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:09:09.933359 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:09.933303 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn podName:5488015e-fad7-4f27-b7a0-62d9bb3d7da0 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:25.933289199 +0000 UTC m=+34.244428167 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-t5pnn" (UniqueName: "kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn") pod "network-check-target-nhnmr" (UID: "5488015e-fad7-4f27-b7a0-62d9bb3d7da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:09:10.039555 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:10.039471 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:10.039712 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:10.039481 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:09:10.039712 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:10.039600 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mbtvm" podUID="a3db17e5-7acd-462b-a6f2-b11617619628" Jun 15 03:09:10.039712 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:10.039614 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:10.039906 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:10.039716 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vz9g7" podUID="c85b7d1f-4d4b-4fdd-9427-d1c25942aebe" Jun 15 03:09:10.039906 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:10.039803 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nhnmr" podUID="5488015e-fad7-4f27-b7a0-62d9bb3d7da0" Jun 15 03:09:12.040061 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:12.040028 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:12.040482 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:12.040163 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mbtvm" podUID="a3db17e5-7acd-462b-a6f2-b11617619628" Jun 15 03:09:12.040482 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:12.040204 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:12.040482 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:12.040287 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nhnmr" podUID="5488015e-fad7-4f27-b7a0-62d9bb3d7da0" Jun 15 03:09:12.040631 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:12.040074 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:09:12.040631 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:12.040612 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vz9g7" podUID="c85b7d1f-4d4b-4fdd-9427-d1c25942aebe" Jun 15 03:09:14.039704 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:14.039536 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:14.040445 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:14.039538 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:14.040445 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:14.039571 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:09:14.040445 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:14.039773 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nhnmr" podUID="5488015e-fad7-4f27-b7a0-62d9bb3d7da0" Jun 15 03:09:14.040445 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:14.039925 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mbtvm" podUID="a3db17e5-7acd-462b-a6f2-b11617619628" Jun 15 03:09:14.040445 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:14.040030 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vz9g7" podUID="c85b7d1f-4d4b-4fdd-9427-d1c25942aebe" Jun 15 03:09:14.146933 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:14.146901 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xhccz" event={"ID":"7fe3e8d4-b5da-4511-a7f8-f28596c5f336","Type":"ContainerStarted","Data":"d807cdce460f1a7721f508c1a3d918874936c345246ea2612f07c2ff49ee5b6f"} Jun 15 03:09:14.149036 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:14.149010 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" event={"ID":"789deb33-3a27-4d7f-881f-31f889f46e0e","Type":"ContainerStarted","Data":"c69d6b5670646d5ced9b64aa6855230a7deede620aa897a9adf33179879e207c"} Jun 15 03:09:14.149140 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:14.149070 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" event={"ID":"789deb33-3a27-4d7f-881f-31f889f46e0e","Type":"ContainerStarted","Data":"b2eb9dcb160435b4a33fa895ac0548ffd3a111aa51fe231aac3981dd8e21241a"} Jun 15 03:09:14.150356 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:14.150329 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s4gh2" event={"ID":"0a161380-ecd7-4473-9cc2-e3041ad5571d","Type":"ContainerStarted","Data":"c945a5f65cffaa64c698717d42258f30d0241f99a894a56fca8d7aecc9eaf406"} Jun 15 03:09:14.151751 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:14.151729 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tpvc8" event={"ID":"d75956ab-4288-4724-a60e-f55e2cb6c932","Type":"ContainerStarted","Data":"1724ccaa4a3022169cea50d7a448ea60c10d658247cdbc04295aa82b81f588cf"} Jun 15 03:09:14.153118 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:14.153094 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lj57j" event={"ID":"ff9255c7-dc08-4be9-9087-b749284c6358","Type":"ContainerStarted","Data":"8fd61539bd895ab4c2156e2a09888223fa512410cfd3eeadbd229aff94c4a1bd"} Jun 15 03:09:14.154557 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:14.154537 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" event={"ID":"80ab5903-b44d-4517-8c0e-fe4abf44d298","Type":"ContainerStarted","Data":"2d17a76cec6a51bad039afd8167be243b7f1d7958f812420b1659105005f0b87"} Jun 15 03:09:14.155998 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:14.155980 2562 generic.go:356] "Generic (PLEG): container finished" podID="28f12465-f135-4586-a19f-98430d1f26b0" containerID="f7b8cd74817634ae19b8fdc41c3ab1bea4ce71627a7b7220558cf7bb96c254d9" exitCode=0 Jun 15 03:09:14.156099 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:14.156046 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn582" event={"ID":"28f12465-f135-4586-a19f-98430d1f26b0","Type":"ContainerDied","Data":"f7b8cd74817634ae19b8fdc41c3ab1bea4ce71627a7b7220558cf7bb96c254d9"} Jun 15 03:09:14.157235 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:14.157155 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-t98lm" event={"ID":"795cf261-afbb-4f3f-84a1-617bc13d25f9","Type":"ContainerStarted","Data":"d78462625c41df8d26ed9e15c3c7b921e6b46274e0f4bba2b3ac9365c33e34f0"} Jun 15 03:09:14.159703 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:14.159673 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xhccz" podStartSLOduration=3.394921507 podStartE2EDuration="22.159660934s" totalImagesPullingTime="18.764739427s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:14.159406233 +0000 UTC m=+22.470545222" watchObservedRunningTime="2026-06-15 03:09:14.159660934 +0000 UTC m=+22.470799938" Jun 15 03:09:14.173999 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:14.173957 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tpvc8" podStartSLOduration=3.359953933 podStartE2EDuration="22.173948505s" totalImagesPullingTime="18.813994572s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:14.173753052 +0000 UTC m=+22.484892067" watchObservedRunningTime="2026-06-15 03:09:14.173948505 +0000 UTC m=+22.485087496" Jun 15 03:09:14.185914 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:14.185872 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lj57j" podStartSLOduration=3.404220042 podStartE2EDuration="22.185859786s" totalImagesPullingTime="18.781639744s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:14.185547955 +0000 UTC m=+22.496686945" watchObservedRunningTime="2026-06-15 03:09:14.185859786 +0000 UTC m=+22.496998776" Jun 15 03:09:14.201060 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:14.201019 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-t98lm" podStartSLOduration=3.4053058800000002 podStartE2EDuration="22.201003875s" totalImagesPullingTime="18.795697995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:14.200455252 +0000 UTC m=+22.511594243" watchObservedRunningTime="2026-06-15 03:09:14.201003875 +0000 UTC m=+22.512142862" Jun 15 03:09:14.236923 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:14.236882 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-s4gh2" podStartSLOduration=11.77507763 podStartE2EDuration="22.236866511s" totalImagesPullingTime="10.461788881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:14.236245055 +0000 UTC m=+22.547384044" watchObservedRunningTime="2026-06-15 03:09:14.236866511 +0000 UTC m=+22.548005500" Jun 15 03:09:15.043638 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:15.043607 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 15 03:09:15.160994 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:15.160916 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ljsf5" event={"ID":"49cf62c9-8b96-43a4-afa7-ac210e06fb8e","Type":"ContainerStarted","Data":"552699cde6040ad4b2ad333b607384097553e6e4c5a99683f448e4b9a7aa9f2e"} Jun 15 03:09:15.162975 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:15.162947 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" event={"ID":"80ab5903-b44d-4517-8c0e-fe4abf44d298","Type":"ContainerStarted","Data":"5a7836a23ee283958cab9f63d6a66b66c761e4398147e9bafdaee750e67b0c75"} Jun 15 03:09:15.169112 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:15.169079 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" event={"ID":"789deb33-3a27-4d7f-881f-31f889f46e0e","Type":"ContainerStarted","Data":"4475851bf8ab90dd6ae5b3cbfb2b6cae5c8314f7086bd202e70570fe29363b8a"} Jun 15 03:09:15.169223 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:15.169183 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" event={"ID":"789deb33-3a27-4d7f-881f-31f889f46e0e","Type":"ContainerStarted","Data":"3f2b3e91e00d12c22f6d940ef77775834114d84409ace8c7eed506cd77d51dbf"} Jun 15 03:09:15.169280 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:15.169234 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" event={"ID":"789deb33-3a27-4d7f-881f-31f889f46e0e","Type":"ContainerStarted","Data":"eccf44aa711fea01e1c47d325daacb05f320e6a8a230d94929f3bb32b2e33606"} Jun 15 03:09:15.169280 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:15.169276 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" event={"ID":"789deb33-3a27-4d7f-881f-31f889f46e0e","Type":"ContainerStarted","Data":"80ec817f7fdbfe5983da3c4ee18fe3b67ae496fec08bb34f0a8400e94f8dbe81"} Jun 15 03:09:15.175284 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:15.175248 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ljsf5" podStartSLOduration=4.398434003 podStartE2EDuration="23.175236861s" totalImagesPullingTime="18.776802858s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:15.174943022 +0000 UTC m=+23.486082014" watchObservedRunningTime="2026-06-15 03:09:15.175236861 +0000 UTC m=+23.486375851" Jun 15 03:09:16.040268 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:16.040229 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:16.040538 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:16.040229 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:09:16.040538 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:16.040415 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mbtvm" podUID="a3db17e5-7acd-462b-a6f2-b11617619628" Jun 15 03:09:16.040538 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:16.040229 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:16.040538 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:16.040488 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vz9g7" podUID="c85b7d1f-4d4b-4fdd-9427-d1c25942aebe" Jun 15 03:09:16.040768 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:16.040564 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nhnmr" podUID="5488015e-fad7-4f27-b7a0-62d9bb3d7da0" Jun 15 03:09:16.042278 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:16.042183 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-15T03:09:15.043632421Z","UUID":"d4a6669b-7434-42df-a220-9f3e83f4c05a","Handler":null,"Name":"","Endpoint":""} Jun 15 03:09:16.043762 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:16.043743 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 15 03:09:16.044148 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:16.043784 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 15 03:09:17.063366 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:17.063177 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-s4gh2" Jun 15 03:09:17.063773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:17.063712 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-s4gh2" Jun 15 03:09:17.176374 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:17.176343 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" event={"ID":"80ab5903-b44d-4517-8c0e-fe4abf44d298","Type":"ContainerStarted","Data":"c33cd82eb00b7854eab7be1aee2ca953fdcd956dbf8c216cb097723d1239ade6"} Jun 15 03:09:17.180328 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:17.180299 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" event={"ID":"789deb33-3a27-4d7f-881f-31f889f46e0e","Type":"ContainerStarted","Data":"1be2c3409e2b2f85fb6f974eabeccb188a7b3651fee270e98c0633a95dd29839"} Jun 15 03:09:17.180532 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:17.180484 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-s4gh2" Jun 15 03:09:17.181029 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:17.181011 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-s4gh2" Jun 15 03:09:17.194185 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:17.194139 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-cfz4z" podStartSLOduration=4.220723325 podStartE2EDuration="25.19412491s" totalImagesPullingTime="20.973401585s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:17.193644723 +0000 UTC m=+25.504783714" watchObservedRunningTime="2026-06-15 03:09:17.19412491 +0000 UTC m=+25.505263938" Jun 15 03:09:18.040353 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:18.040318 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:18.040353 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:18.040335 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:09:18.040595 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:18.040447 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mbtvm" podUID="a3db17e5-7acd-462b-a6f2-b11617619628" Jun 15 03:09:18.040595 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:18.040477 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:18.040595 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:18.040570 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nhnmr" podUID="5488015e-fad7-4f27-b7a0-62d9bb3d7da0" Jun 15 03:09:18.040731 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:18.040657 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vz9g7" podUID="c85b7d1f-4d4b-4fdd-9427-d1c25942aebe" Jun 15 03:09:20.039916 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:20.039720 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:20.040514 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:20.039752 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:09:20.040514 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:20.040011 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mbtvm" podUID="a3db17e5-7acd-462b-a6f2-b11617619628" Jun 15 03:09:20.040514 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:20.040101 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vz9g7" podUID="c85b7d1f-4d4b-4fdd-9427-d1c25942aebe" Jun 15 03:09:20.040514 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:20.039859 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:20.040514 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:20.040199 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nhnmr" podUID="5488015e-fad7-4f27-b7a0-62d9bb3d7da0" Jun 15 03:09:20.191863 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:20.191735 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" event={"ID":"789deb33-3a27-4d7f-881f-31f889f46e0e","Type":"ContainerStarted","Data":"c92b5b4012822c8dedba8dc2b823da649da2598106dc576367cbbda1f2899f46"} Jun 15 03:09:20.192131 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:20.192052 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:09:20.192131 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:20.192120 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:09:20.207727 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:20.207701 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:09:20.214520 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:20.214480 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" podStartSLOduration=9.258983815 podStartE2EDuration="28.214467722s" totalImagesPullingTime="18.955483907s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:20.213287517 +0000 UTC m=+28.524426507" watchObservedRunningTime="2026-06-15 03:09:20.214467722 +0000 UTC m=+28.525606720" Jun 15 03:09:21.197414 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:21.197375 2562 generic.go:356] "Generic (PLEG): container finished" podID="28f12465-f135-4586-a19f-98430d1f26b0" containerID="07a09f542bd158cdfa6e1955cc85e2774cc2e766531b5f3add87fc5c474ecf2a" exitCode=0 Jun 15 03:09:21.198054 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:21.197455 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn582" event={"ID":"28f12465-f135-4586-a19f-98430d1f26b0","Type":"ContainerDied","Data":"07a09f542bd158cdfa6e1955cc85e2774cc2e766531b5f3add87fc5c474ecf2a"} Jun 15 03:09:21.198129 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:21.198091 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:09:21.216303 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:21.216269 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:09:21.932989 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:21.932792 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mbtvm"] Jun 15 03:09:21.933141 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:21.933069 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nhnmr"] Jun 15 03:09:21.933141 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:21.933092 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:21.933210 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:21.933188 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mbtvm" podUID="a3db17e5-7acd-462b-a6f2-b11617619628" Jun 15 03:09:21.933255 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:21.933239 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:21.933363 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:21.933346 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nhnmr" podUID="5488015e-fad7-4f27-b7a0-62d9bb3d7da0" Jun 15 03:09:21.944972 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:21.944947 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vz9g7"] Jun 15 03:09:21.945103 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:21.945064 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:09:21.945196 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:21.945160 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vz9g7" podUID="c85b7d1f-4d4b-4fdd-9427-d1c25942aebe" Jun 15 03:09:23.039936 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:23.039860 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:23.040344 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:23.039965 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nhnmr" podUID="5488015e-fad7-4f27-b7a0-62d9bb3d7da0" Jun 15 03:09:23.203870 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:23.203819 2562 generic.go:356] "Generic (PLEG): container finished" podID="28f12465-f135-4586-a19f-98430d1f26b0" containerID="79a1dfe00efeb179b7517be759056d75cf236ed685708a76eaa440c05d23af14" exitCode=0 Jun 15 03:09:23.204027 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:23.203881 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn582" event={"ID":"28f12465-f135-4586-a19f-98430d1f26b0","Type":"ContainerDied","Data":"79a1dfe00efeb179b7517be759056d75cf236ed685708a76eaa440c05d23af14"} Jun 15 03:09:24.039684 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:24.039655 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:24.039840 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:24.039792 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mbtvm" podUID="a3db17e5-7acd-462b-a6f2-b11617619628" Jun 15 03:09:24.039880 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:24.039854 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:09:24.039966 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:24.039950 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vz9g7" podUID="c85b7d1f-4d4b-4fdd-9427-d1c25942aebe" Jun 15 03:09:25.039848 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:25.039797 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:25.040030 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:25.039916 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nhnmr" podUID="5488015e-fad7-4f27-b7a0-62d9bb3d7da0" Jun 15 03:09:25.210931 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:25.210898 2562 generic.go:356] "Generic (PLEG): container finished" podID="28f12465-f135-4586-a19f-98430d1f26b0" containerID="a20ee6f3310579d52d36107ac7a0548913d5e237233beabe10c678a50609f933" exitCode=0 Jun 15 03:09:25.211094 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:25.210941 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn582" event={"ID":"28f12465-f135-4586-a19f-98430d1f26b0","Type":"ContainerDied","Data":"a20ee6f3310579d52d36107ac7a0548913d5e237233beabe10c678a50609f933"} Jun 15 03:09:25.750119 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:25.750076 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs\") pod \"network-metrics-daemon-vz9g7\" (UID: \"c85b7d1f-4d4b-4fdd-9427-d1c25942aebe\") " pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:09:25.750322 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:25.750245 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:09:25.750388 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:25.750339 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs podName:c85b7d1f-4d4b-4fdd-9427-d1c25942aebe nodeName:}" failed. No retries permitted until 2026-06-15 03:09:57.750318226 +0000 UTC m=+66.061457197 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs") pod "network-metrics-daemon-vz9g7" (UID: "c85b7d1f-4d4b-4fdd-9427-d1c25942aebe") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 15 03:09:25.850809 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:25.850775 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret\") pod \"global-pull-secret-syncer-mbtvm\" (UID: \"a3db17e5-7acd-462b-a6f2-b11617619628\") " pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:25.850983 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:25.850941 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 15 03:09:25.851058 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:25.851020 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret podName:a3db17e5-7acd-462b-a6f2-b11617619628 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:57.851001315 +0000 UTC m=+66.162140283 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret") pod "global-pull-secret-syncer-mbtvm" (UID: "a3db17e5-7acd-462b-a6f2-b11617619628") : object "kube-system"/"original-pull-secret" not registered Jun 15 03:09:25.951501 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:25.951463 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5pnn\" (UniqueName: \"kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn\") pod \"network-check-target-nhnmr\" (UID: \"5488015e-fad7-4f27-b7a0-62d9bb3d7da0\") " pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:25.951688 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:25.951654 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 15 03:09:25.951688 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:25.951680 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 15 03:09:25.951795 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:25.951693 2562 projected.go:196] Error preparing data for projected volume kube-api-access-t5pnn for pod openshift-network-diagnostics/network-check-target-nhnmr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:09:25.951795 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:25.951761 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn podName:5488015e-fad7-4f27-b7a0-62d9bb3d7da0 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:57.951741404 +0000 UTC m=+66.262880385 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-t5pnn" (UniqueName: "kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn") pod "network-check-target-nhnmr" (UID: "5488015e-fad7-4f27-b7a0-62d9bb3d7da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 15 03:09:26.040185 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.040111 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:09:26.040662 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.040111 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:26.040662 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:26.040230 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vz9g7" podUID="c85b7d1f-4d4b-4fdd-9427-d1c25942aebe" Jun 15 03:09:26.040662 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:26.040311 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mbtvm" podUID="a3db17e5-7acd-462b-a6f2-b11617619628" Jun 15 03:09:26.522934 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.522853 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeReady" Jun 15 03:09:26.523094 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.523022 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 15 03:09:26.574121 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.574077 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nb5km"] Jun 15 03:09:26.578460 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.578436 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-p9h4n"] Jun 15 03:09:26.578592 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.578560 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nb5km" Jun 15 03:09:26.580916 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.580891 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 15 03:09:26.581036 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.580900 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b8jdx\"" Jun 15 03:09:26.581036 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.580991 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 15 03:09:26.582575 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.582558 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p9h4n" Jun 15 03:09:26.585637 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.585444 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 15 03:09:26.585637 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.585471 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 15 03:09:26.585637 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.585498 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dqgpx\"" Jun 15 03:09:26.585894 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.585878 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 15 03:09:26.588406 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.588383 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p9h4n"] Jun 15 03:09:26.591382 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.591327 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nb5km"] Jun 15 03:09:26.757696 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.757662 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htmq9\" (UniqueName: \"kubernetes.io/projected/f61b9993-27b1-4a1d-b7bc-632802832528-kube-api-access-htmq9\") pod \"ingress-canary-p9h4n\" (UID: \"f61b9993-27b1-4a1d-b7bc-632802832528\") " pod="openshift-ingress-canary/ingress-canary-p9h4n" Jun 15 03:09:26.757696 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.757696 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-tmp-dir\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:09:26.757941 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.757785 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:09:26.757941 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.757891 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-config-volume\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:09:26.757941 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.757921 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnqsx\" (UniqueName: \"kubernetes.io/projected/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-kube-api-access-hnqsx\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:09:26.758079 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.757951 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f61b9993-27b1-4a1d-b7bc-632802832528-cert\") pod \"ingress-canary-p9h4n\" (UID: \"f61b9993-27b1-4a1d-b7bc-632802832528\") " pod="openshift-ingress-canary/ingress-canary-p9h4n" Jun 15 03:09:26.859067 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.858991 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:09:26.859067 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.859053 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-config-volume\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:09:26.859291 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.859070 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnqsx\" (UniqueName: \"kubernetes.io/projected/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-kube-api-access-hnqsx\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:09:26.859291 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.859090 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f61b9993-27b1-4a1d-b7bc-632802832528-cert\") pod \"ingress-canary-p9h4n\" (UID: \"f61b9993-27b1-4a1d-b7bc-632802832528\") " pod="openshift-ingress-canary/ingress-canary-p9h4n" Jun 15 03:09:26.859291 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.859116 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htmq9\" (UniqueName: \"kubernetes.io/projected/f61b9993-27b1-4a1d-b7bc-632802832528-kube-api-access-htmq9\") pod \"ingress-canary-p9h4n\" (UID: \"f61b9993-27b1-4a1d-b7bc-632802832528\") " pod="openshift-ingress-canary/ingress-canary-p9h4n" Jun 15 03:09:26.859291 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.859139 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-tmp-dir\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:09:26.859291 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:26.859151 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 03:09:26.859291 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:26.859233 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls podName:167a4bfe-f33d-4a1e-878c-8faebc6f61e7 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:27.359211149 +0000 UTC m=+35.670350130 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls") pod "dns-default-nb5km" (UID: "167a4bfe-f33d-4a1e-878c-8faebc6f61e7") : secret "dns-default-metrics-tls" not found Jun 15 03:09:26.859619 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.859528 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-tmp-dir\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:09:26.859739 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.859712 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-config-volume\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:09:26.863888 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.863867 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f61b9993-27b1-4a1d-b7bc-632802832528-cert\") pod \"ingress-canary-p9h4n\" (UID: \"f61b9993-27b1-4a1d-b7bc-632802832528\") " pod="openshift-ingress-canary/ingress-canary-p9h4n" Jun 15 03:09:26.867400 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.867376 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnqsx\" (UniqueName: \"kubernetes.io/projected/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-kube-api-access-hnqsx\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:09:26.867632 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.867617 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htmq9\" (UniqueName: \"kubernetes.io/projected/f61b9993-27b1-4a1d-b7bc-632802832528-kube-api-access-htmq9\") pod \"ingress-canary-p9h4n\" (UID: \"f61b9993-27b1-4a1d-b7bc-632802832528\") " pod="openshift-ingress-canary/ingress-canary-p9h4n" Jun 15 03:09:26.898699 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:26.898651 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p9h4n" Jun 15 03:09:27.040259 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:27.040058 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:27.042575 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:27.042553 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 15 03:09:27.042693 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:27.042633 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 15 03:09:27.042693 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:27.042672 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hxgvb\"" Jun 15 03:09:27.051972 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:27.051935 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p9h4n"] Jun 15 03:09:27.056303 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:09:27.056282 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf61b9993_27b1_4a1d_b7bc_632802832528.slice/crio-704311d79dc88cd3904c1db7973915aa07b1856bb3d192a40c114117d15ee6c6 WatchSource:0}: Error finding container 704311d79dc88cd3904c1db7973915aa07b1856bb3d192a40c114117d15ee6c6: Status 404 returned error can't find the container with id 704311d79dc88cd3904c1db7973915aa07b1856bb3d192a40c114117d15ee6c6 Jun 15 03:09:27.221701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:27.221572 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p9h4n" event={"ID":"f61b9993-27b1-4a1d-b7bc-632802832528","Type":"ContainerStarted","Data":"704311d79dc88cd3904c1db7973915aa07b1856bb3d192a40c114117d15ee6c6"} Jun 15 03:09:27.362489 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:27.362455 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:09:27.362666 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:27.362640 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 03:09:27.362749 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:27.362719 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls podName:167a4bfe-f33d-4a1e-878c-8faebc6f61e7 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:28.362699228 +0000 UTC m=+36.673838196 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls") pod "dns-default-nb5km" (UID: "167a4bfe-f33d-4a1e-878c-8faebc6f61e7") : secret "dns-default-metrics-tls" not found Jun 15 03:09:28.040398 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:28.040365 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:28.040862 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:28.040365 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:09:28.043105 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:28.043062 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 15 03:09:28.043105 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:28.043096 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 15 03:09:28.043698 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:28.043680 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lnpms\"" Jun 15 03:09:28.370266 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:28.370180 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:09:28.370432 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:28.370330 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 03:09:28.370432 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:28.370399 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls podName:167a4bfe-f33d-4a1e-878c-8faebc6f61e7 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:30.370382399 +0000 UTC m=+38.681521367 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls") pod "dns-default-nb5km" (UID: "167a4bfe-f33d-4a1e-878c-8faebc6f61e7") : secret "dns-default-metrics-tls" not found Jun 15 03:09:30.231383 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:30.231212 2562 generic.go:356] "Generic (PLEG): container finished" podID="28f12465-f135-4586-a19f-98430d1f26b0" containerID="eba415643e2f9c615981e96cf2c28ffa9719451e09c6fe326af27d6b5c7f41d4" exitCode=0 Jun 15 03:09:30.231740 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:30.231251 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn582" event={"ID":"28f12465-f135-4586-a19f-98430d1f26b0","Type":"ContainerDied","Data":"eba415643e2f9c615981e96cf2c28ffa9719451e09c6fe326af27d6b5c7f41d4"} Jun 15 03:09:30.232925 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:30.232903 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p9h4n" event={"ID":"f61b9993-27b1-4a1d-b7bc-632802832528","Type":"ContainerStarted","Data":"01c55236489a3bb1b7a69d9802209349bd3c9a88461e731a650d3112548adf07"} Jun 15 03:09:30.386417 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:30.386388 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:09:30.386560 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:30.386540 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 03:09:30.386627 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:30.386603 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls podName:167a4bfe-f33d-4a1e-878c-8faebc6f61e7 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:34.38658482 +0000 UTC m=+42.697723791 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls") pod "dns-default-nb5km" (UID: "167a4bfe-f33d-4a1e-878c-8faebc6f61e7") : secret "dns-default-metrics-tls" not found Jun 15 03:09:31.238365 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:31.238334 2562 generic.go:356] "Generic (PLEG): container finished" podID="28f12465-f135-4586-a19f-98430d1f26b0" containerID="e8a945bb6f1fc24361a250fae17d403857c27131fd3b9c32aaaf52d54768dd6a" exitCode=0 Jun 15 03:09:31.238913 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:31.238419 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn582" event={"ID":"28f12465-f135-4586-a19f-98430d1f26b0","Type":"ContainerDied","Data":"e8a945bb6f1fc24361a250fae17d403857c27131fd3b9c32aaaf52d54768dd6a"} Jun 15 03:09:31.259677 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:31.259634 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-p9h4n" podStartSLOduration=2.570687105 podStartE2EDuration="5.259620074s" totalImagesPullingTime="2.688932969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:09:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:30.268660689 +0000 UTC m=+38.579799680" watchObservedRunningTime="2026-06-15 03:09:31.259620074 +0000 UTC m=+39.570759092" Jun 15 03:09:32.244306 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:32.244270 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pn582" event={"ID":"28f12465-f135-4586-a19f-98430d1f26b0","Type":"ContainerStarted","Data":"bc2e1b209613bc9fcc2135dfc6150f5de80e69ce39e1c58e8754a595e6654ee1"} Jun 15 03:09:32.267906 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:32.267854 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pn582" podStartSLOduration=9.028214868 podStartE2EDuration="40.267822754s" totalImagesPullingTime="31.239607886s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:32.266489362 +0000 UTC m=+40.577628351" watchObservedRunningTime="2026-06-15 03:09:32.267822754 +0000 UTC m=+40.578961744" Jun 15 03:09:34.417386 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:34.417346 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:09:34.417775 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:34.417458 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 03:09:34.417775 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:34.417519 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls podName:167a4bfe-f33d-4a1e-878c-8faebc6f61e7 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:42.417504699 +0000 UTC m=+50.728643667 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls") pod "dns-default-nb5km" (UID: "167a4bfe-f33d-4a1e-878c-8faebc6f61e7") : secret "dns-default-metrics-tls" not found Jun 15 03:09:42.469355 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:42.469315 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:09:42.469815 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:42.469466 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 03:09:42.469815 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:42.469544 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls podName:167a4bfe-f33d-4a1e-878c-8faebc6f61e7 nodeName:}" failed. No retries permitted until 2026-06-15 03:09:58.469527339 +0000 UTC m=+66.780666307 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls") pod "dns-default-nb5km" (UID: "167a4bfe-f33d-4a1e-878c-8faebc6f61e7") : secret "dns-default-metrics-tls" not found Jun 15 03:09:51.567130 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.567096 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w"] Jun 15 03:09:51.594233 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.594203 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w"] Jun 15 03:09:51.594398 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.594330 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:09:51.596371 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.596349 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 15 03:09:51.596484 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.596350 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 15 03:09:51.596634 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.596616 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jun 15 03:09:51.596634 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.596635 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 15 03:09:51.596758 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.596693 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jun 15 03:09:51.596758 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.596713 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jun 15 03:09:51.597425 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.597409 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jun 15 03:09:51.635300 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.635271 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/193678ba-2d80-4a74-a734-4cbc938e653f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-76c56fdcc-xtl2w\" (UID: \"193678ba-2d80-4a74-a734-4cbc938e653f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:09:51.635438 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.635381 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6tp6\" (UniqueName: \"kubernetes.io/projected/193678ba-2d80-4a74-a734-4cbc938e653f-kube-api-access-x6tp6\") pod \"cluster-proxy-proxy-agent-76c56fdcc-xtl2w\" (UID: \"193678ba-2d80-4a74-a734-4cbc938e653f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:09:51.635438 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.635421 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/193678ba-2d80-4a74-a734-4cbc938e653f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-76c56fdcc-xtl2w\" (UID: \"193678ba-2d80-4a74-a734-4cbc938e653f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:09:51.635512 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.635457 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/193678ba-2d80-4a74-a734-4cbc938e653f-hub\") pod \"cluster-proxy-proxy-agent-76c56fdcc-xtl2w\" (UID: \"193678ba-2d80-4a74-a734-4cbc938e653f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:09:51.635512 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.635483 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/193678ba-2d80-4a74-a734-4cbc938e653f-ca\") pod \"cluster-proxy-proxy-agent-76c56fdcc-xtl2w\" (UID: \"193678ba-2d80-4a74-a734-4cbc938e653f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:09:51.635512 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.635498 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/193678ba-2d80-4a74-a734-4cbc938e653f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-76c56fdcc-xtl2w\" (UID: \"193678ba-2d80-4a74-a734-4cbc938e653f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:09:51.736718 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.736677 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/193678ba-2d80-4a74-a734-4cbc938e653f-ca\") pod \"cluster-proxy-proxy-agent-76c56fdcc-xtl2w\" (UID: \"193678ba-2d80-4a74-a734-4cbc938e653f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:09:51.736718 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.736717 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/193678ba-2d80-4a74-a734-4cbc938e653f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-76c56fdcc-xtl2w\" (UID: \"193678ba-2d80-4a74-a734-4cbc938e653f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:09:51.736959 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.736767 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/193678ba-2d80-4a74-a734-4cbc938e653f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-76c56fdcc-xtl2w\" (UID: \"193678ba-2d80-4a74-a734-4cbc938e653f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:09:51.736959 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.736808 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6tp6\" (UniqueName: \"kubernetes.io/projected/193678ba-2d80-4a74-a734-4cbc938e653f-kube-api-access-x6tp6\") pod \"cluster-proxy-proxy-agent-76c56fdcc-xtl2w\" (UID: \"193678ba-2d80-4a74-a734-4cbc938e653f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:09:51.736959 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.736950 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/193678ba-2d80-4a74-a734-4cbc938e653f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-76c56fdcc-xtl2w\" (UID: \"193678ba-2d80-4a74-a734-4cbc938e653f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:09:51.737091 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.737004 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/193678ba-2d80-4a74-a734-4cbc938e653f-hub\") pod \"cluster-proxy-proxy-agent-76c56fdcc-xtl2w\" (UID: \"193678ba-2d80-4a74-a734-4cbc938e653f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:09:51.737560 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.737535 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/193678ba-2d80-4a74-a734-4cbc938e653f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-76c56fdcc-xtl2w\" (UID: \"193678ba-2d80-4a74-a734-4cbc938e653f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:09:51.740707 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.740682 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/193678ba-2d80-4a74-a734-4cbc938e653f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-76c56fdcc-xtl2w\" (UID: \"193678ba-2d80-4a74-a734-4cbc938e653f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:09:51.740793 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.740749 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/193678ba-2d80-4a74-a734-4cbc938e653f-ca\") pod \"cluster-proxy-proxy-agent-76c56fdcc-xtl2w\" (UID: \"193678ba-2d80-4a74-a734-4cbc938e653f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:09:51.740926 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.740911 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/193678ba-2d80-4a74-a734-4cbc938e653f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-76c56fdcc-xtl2w\" (UID: \"193678ba-2d80-4a74-a734-4cbc938e653f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:09:51.740972 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.740945 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/193678ba-2d80-4a74-a734-4cbc938e653f-hub\") pod \"cluster-proxy-proxy-agent-76c56fdcc-xtl2w\" (UID: \"193678ba-2d80-4a74-a734-4cbc938e653f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:09:51.743902 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.743878 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6tp6\" (UniqueName: \"kubernetes.io/projected/193678ba-2d80-4a74-a734-4cbc938e653f-kube-api-access-x6tp6\") pod \"cluster-proxy-proxy-agent-76c56fdcc-xtl2w\" (UID: \"193678ba-2d80-4a74-a734-4cbc938e653f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:09:51.912685 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:51.912654 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:09:52.057913 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:52.057885 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w"] Jun 15 03:09:52.060940 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:09:52.060907 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod193678ba_2d80_4a74_a734_4cbc938e653f.slice/crio-d915da0330b17af49318a6ba78a9cbb8c2e7267fddcea182df29aaf80bec6b35 WatchSource:0}: Error finding container d915da0330b17af49318a6ba78a9cbb8c2e7267fddcea182df29aaf80bec6b35: Status 404 returned error can't find the container with id d915da0330b17af49318a6ba78a9cbb8c2e7267fddcea182df29aaf80bec6b35 Jun 15 03:09:52.291074 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:52.290987 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" event={"ID":"193678ba-2d80-4a74-a734-4cbc938e653f","Type":"ContainerStarted","Data":"d915da0330b17af49318a6ba78a9cbb8c2e7267fddcea182df29aaf80bec6b35"} Jun 15 03:09:53.216717 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:53.216690 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9r9mn" Jun 15 03:09:56.303897 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:56.303865 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" event={"ID":"193678ba-2d80-4a74-a734-4cbc938e653f","Type":"ContainerStarted","Data":"99a9fb0721d26f993721030cdaa2cbc2e43933bb2c715a8ad73a4bbba0a69566"} Jun 15 03:09:56.304178 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:56.303919 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" event={"ID":"193678ba-2d80-4a74-a734-4cbc938e653f","Type":"ContainerStarted","Data":"8131e86835a4b4380a4492bbd197b85b9cf6d6bc5a41a7bcd13b9e55bd4270b6"} Jun 15 03:09:56.304178 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:56.303938 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" event={"ID":"193678ba-2d80-4a74-a734-4cbc938e653f","Type":"ContainerStarted","Data":"fa340150c50c27f9a2d4deee4293f75835cb826b1e3298b9972b94209eaad10e"} Jun 15 03:09:57.327612 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:57.327561 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" podStartSLOduration=2.262772663 podStartE2EDuration="6.327545612s" totalImagesPullingTime="4.064772949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:09:57.327217793 +0000 UTC m=+65.638356804" watchObservedRunningTime="2026-06-15 03:09:57.327545612 +0000 UTC m=+65.638684601" Jun 15 03:09:57.780848 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:57.780793 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs\") pod \"network-metrics-daemon-vz9g7\" (UID: \"c85b7d1f-4d4b-4fdd-9427-d1c25942aebe\") " pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:09:57.783257 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:57.783239 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 15 03:09:57.791929 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:57.791908 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 15 03:09:57.791981 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:57.791969 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs podName:c85b7d1f-4d4b-4fdd-9427-d1c25942aebe nodeName:}" failed. No retries permitted until 2026-06-15 03:11:01.79195293 +0000 UTC m=+130.103091898 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs") pod "network-metrics-daemon-vz9g7" (UID: "c85b7d1f-4d4b-4fdd-9427-d1c25942aebe") : secret "metrics-daemon-secret" not found Jun 15 03:09:57.881844 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:57.881788 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret\") pod \"global-pull-secret-syncer-mbtvm\" (UID: \"a3db17e5-7acd-462b-a6f2-b11617619628\") " pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:57.884235 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:57.884217 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 15 03:09:57.895057 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:57.895030 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3db17e5-7acd-462b-a6f2-b11617619628-original-pull-secret\") pod \"global-pull-secret-syncer-mbtvm\" (UID: \"a3db17e5-7acd-462b-a6f2-b11617619628\") " pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:57.982893 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:57.982823 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5pnn\" (UniqueName: \"kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn\") pod \"network-check-target-nhnmr\" (UID: \"5488015e-fad7-4f27-b7a0-62d9bb3d7da0\") " pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:57.985281 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:57.985263 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 15 03:09:57.995930 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:57.995906 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 15 03:09:58.006868 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:58.006828 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5pnn\" (UniqueName: \"kubernetes.io/projected/5488015e-fad7-4f27-b7a0-62d9bb3d7da0-kube-api-access-t5pnn\") pod \"network-check-target-nhnmr\" (UID: \"5488015e-fad7-4f27-b7a0-62d9bb3d7da0\") " pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:58.053186 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:58.053110 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mbtvm" Jun 15 03:09:58.183504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:58.183469 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mbtvm"] Jun 15 03:09:58.187506 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:09:58.187484 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3db17e5_7acd_462b_a6f2_b11617619628.slice/crio-845cc6f122f4e0cc8c03d68b9a2f8d388ad8a6ef7fa1951b2219d4c11f8e8ede WatchSource:0}: Error finding container 845cc6f122f4e0cc8c03d68b9a2f8d388ad8a6ef7fa1951b2219d4c11f8e8ede: Status 404 returned error can't find the container with id 845cc6f122f4e0cc8c03d68b9a2f8d388ad8a6ef7fa1951b2219d4c11f8e8ede Jun 15 03:09:58.253416 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:58.253385 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hxgvb\"" Jun 15 03:09:58.262078 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:58.262061 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:09:58.311369 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:58.311320 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mbtvm" event={"ID":"a3db17e5-7acd-462b-a6f2-b11617619628","Type":"ContainerStarted","Data":"845cc6f122f4e0cc8c03d68b9a2f8d388ad8a6ef7fa1951b2219d4c11f8e8ede"} Jun 15 03:09:58.382936 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:58.382884 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nhnmr"] Jun 15 03:09:58.385805 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:09:58.385766 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5488015e_fad7_4f27_b7a0_62d9bb3d7da0.slice/crio-d6d0bd61af2232d4b76bba5e5f0a677ff7a5e871124a0f9b0b0b70ba32a9dde0 WatchSource:0}: Error finding container d6d0bd61af2232d4b76bba5e5f0a677ff7a5e871124a0f9b0b0b70ba32a9dde0: Status 404 returned error can't find the container with id d6d0bd61af2232d4b76bba5e5f0a677ff7a5e871124a0f9b0b0b70ba32a9dde0 Jun 15 03:09:58.486466 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:58.486437 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:09:58.486631 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:58.486553 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 03:09:58.486631 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:09:58.486602 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls podName:167a4bfe-f33d-4a1e-878c-8faebc6f61e7 nodeName:}" failed. No retries permitted until 2026-06-15 03:10:30.486589653 +0000 UTC m=+98.797728621 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls") pod "dns-default-nb5km" (UID: "167a4bfe-f33d-4a1e-878c-8faebc6f61e7") : secret "dns-default-metrics-tls" not found Jun 15 03:09:59.315801 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:09:59.315761 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nhnmr" event={"ID":"5488015e-fad7-4f27-b7a0-62d9bb3d7da0","Type":"ContainerStarted","Data":"d6d0bd61af2232d4b76bba5e5f0a677ff7a5e871124a0f9b0b0b70ba32a9dde0"} Jun 15 03:10:03.329218 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:10:03.329120 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nhnmr" event={"ID":"5488015e-fad7-4f27-b7a0-62d9bb3d7da0","Type":"ContainerStarted","Data":"c9312c638b024345b41479845c6c84dafd147c9aea21346c4e37e89472fd5325"} Jun 15 03:10:03.329218 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:10:03.329210 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:10:03.330510 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:10:03.330483 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mbtvm" event={"ID":"a3db17e5-7acd-462b-a6f2-b11617619628","Type":"ContainerStarted","Data":"a00956b0d498a92bf4bdeac0e6aefe1a873e6a7acfb29689de84c6faa59718ac"} Jun 15 03:10:03.343873 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:10:03.343804 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nhnmr" podStartSLOduration=66.811717477 podStartE2EDuration="1m11.343791159s" totalImagesPullingTime="4.532073682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:10:03.342968265 +0000 UTC m=+71.654107273" watchObservedRunningTime="2026-06-15 03:10:03.343791159 +0000 UTC m=+71.654930127" Jun 15 03:10:03.355957 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:10:03.355915 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mbtvm" podStartSLOduration=66.622149472 podStartE2EDuration="1m11.355900102s" totalImagesPullingTime="4.73375063s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:10:03.355296843 +0000 UTC m=+71.666435835" watchObservedRunningTime="2026-06-15 03:10:03.355900102 +0000 UTC m=+71.667039091" Jun 15 03:10:30.509685 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:10:30.509646 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:10:30.510314 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:10:30.509790 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 03:10:30.510314 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:10:30.509905 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls podName:167a4bfe-f33d-4a1e-878c-8faebc6f61e7 nodeName:}" failed. No retries permitted until 2026-06-15 03:11:34.509879765 +0000 UTC m=+162.821018749 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls") pod "dns-default-nb5km" (UID: "167a4bfe-f33d-4a1e-878c-8faebc6f61e7") : secret "dns-default-metrics-tls" not found Jun 15 03:10:34.336515 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:10:34.336485 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nhnmr" Jun 15 03:11:01.814195 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:11:01.814148 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs\") pod \"network-metrics-daemon-vz9g7\" (UID: \"c85b7d1f-4d4b-4fdd-9427-d1c25942aebe\") " pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:11:01.814684 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:11:01.814309 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 15 03:11:01.814684 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:11:01.814388 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs podName:c85b7d1f-4d4b-4fdd-9427-d1c25942aebe nodeName:}" failed. No retries permitted until 2026-06-15 03:13:03.814369316 +0000 UTC m=+252.125508291 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs") pod "network-metrics-daemon-vz9g7" (UID: "c85b7d1f-4d4b-4fdd-9427-d1c25942aebe") : secret "metrics-daemon-secret" not found Jun 15 03:11:29.591901 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:11:29.591852 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-nb5km" podUID="167a4bfe-f33d-4a1e-878c-8faebc6f61e7" Jun 15 03:11:30.573493 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:11:30.573465 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nb5km" Jun 15 03:11:31.060494 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:11:31.060449 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-vz9g7" podUID="c85b7d1f-4d4b-4fdd-9427-d1c25942aebe" Jun 15 03:11:34.529933 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:11:34.529878 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:11:34.530412 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:11:34.530029 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 15 03:11:34.530412 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:11:34.530100 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls podName:167a4bfe-f33d-4a1e-878c-8faebc6f61e7 nodeName:}" failed. No retries permitted until 2026-06-15 03:13:36.530082372 +0000 UTC m=+284.841221362 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls") pod "dns-default-nb5km" (UID: "167a4bfe-f33d-4a1e-878c-8faebc6f61e7") : secret "dns-default-metrics-tls" not found Jun 15 03:11:43.040285 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:11:43.040177 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:12:01.250341 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.250303 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5bbd86d6bd-dswhg"] Jun 15 03:12:01.253209 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.253187 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" Jun 15 03:12:01.255516 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.255495 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jun 15 03:12:01.255857 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.255826 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-qcqfq\"" Jun 15 03:12:01.256173 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.256157 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 15 03:12:01.256225 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.256157 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 15 03:12:01.256264 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.256160 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jun 15 03:12:01.265151 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.265131 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jun 15 03:12:01.270460 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.270439 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5bbd86d6bd-dswhg"] Jun 15 03:12:01.322248 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.322214 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06bfd370-abe8-4b3a-a82c-41e7d0ade9c7-service-ca-bundle\") pod \"insights-operator-5bbd86d6bd-dswhg\" (UID: \"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7\") " pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" Jun 15 03:12:01.322406 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.322256 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/06bfd370-abe8-4b3a-a82c-41e7d0ade9c7-snapshots\") pod \"insights-operator-5bbd86d6bd-dswhg\" (UID: \"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7\") " pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" Jun 15 03:12:01.322406 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.322315 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntmxt\" (UniqueName: \"kubernetes.io/projected/06bfd370-abe8-4b3a-a82c-41e7d0ade9c7-kube-api-access-ntmxt\") pod \"insights-operator-5bbd86d6bd-dswhg\" (UID: \"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7\") " pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" Jun 15 03:12:01.322406 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.322348 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06bfd370-abe8-4b3a-a82c-41e7d0ade9c7-trusted-ca-bundle\") pod \"insights-operator-5bbd86d6bd-dswhg\" (UID: \"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7\") " pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" Jun 15 03:12:01.322406 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.322387 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06bfd370-abe8-4b3a-a82c-41e7d0ade9c7-tmp\") pod \"insights-operator-5bbd86d6bd-dswhg\" (UID: \"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7\") " pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" Jun 15 03:12:01.322539 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.322421 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/06bfd370-abe8-4b3a-a82c-41e7d0ade9c7-serving-cert\") pod \"insights-operator-5bbd86d6bd-dswhg\" (UID: \"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7\") " pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" Jun 15 03:12:01.422978 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.422943 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06bfd370-abe8-4b3a-a82c-41e7d0ade9c7-trusted-ca-bundle\") pod \"insights-operator-5bbd86d6bd-dswhg\" (UID: \"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7\") " pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" Jun 15 03:12:01.422978 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.422979 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06bfd370-abe8-4b3a-a82c-41e7d0ade9c7-tmp\") pod \"insights-operator-5bbd86d6bd-dswhg\" (UID: \"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7\") " pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" Jun 15 03:12:01.423207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.423016 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/06bfd370-abe8-4b3a-a82c-41e7d0ade9c7-serving-cert\") pod \"insights-operator-5bbd86d6bd-dswhg\" (UID: \"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7\") " pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" Jun 15 03:12:01.423207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.423038 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06bfd370-abe8-4b3a-a82c-41e7d0ade9c7-service-ca-bundle\") pod \"insights-operator-5bbd86d6bd-dswhg\" (UID: \"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7\") " pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" Jun 15 03:12:01.423207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.423060 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/06bfd370-abe8-4b3a-a82c-41e7d0ade9c7-snapshots\") pod \"insights-operator-5bbd86d6bd-dswhg\" (UID: \"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7\") " pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" Jun 15 03:12:01.423207 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.423083 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntmxt\" (UniqueName: \"kubernetes.io/projected/06bfd370-abe8-4b3a-a82c-41e7d0ade9c7-kube-api-access-ntmxt\") pod \"insights-operator-5bbd86d6bd-dswhg\" (UID: \"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7\") " pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" Jun 15 03:12:01.423458 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.423433 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06bfd370-abe8-4b3a-a82c-41e7d0ade9c7-tmp\") pod \"insights-operator-5bbd86d6bd-dswhg\" (UID: \"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7\") " pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" Jun 15 03:12:01.423628 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.423606 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06bfd370-abe8-4b3a-a82c-41e7d0ade9c7-service-ca-bundle\") pod \"insights-operator-5bbd86d6bd-dswhg\" (UID: \"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7\") " pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" Jun 15 03:12:01.423703 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.423641 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/06bfd370-abe8-4b3a-a82c-41e7d0ade9c7-snapshots\") pod \"insights-operator-5bbd86d6bd-dswhg\" (UID: \"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7\") " pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" Jun 15 03:12:01.423826 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.423807 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06bfd370-abe8-4b3a-a82c-41e7d0ade9c7-trusted-ca-bundle\") pod \"insights-operator-5bbd86d6bd-dswhg\" (UID: \"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7\") " pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" Jun 15 03:12:01.425455 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.425432 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/06bfd370-abe8-4b3a-a82c-41e7d0ade9c7-serving-cert\") pod \"insights-operator-5bbd86d6bd-dswhg\" (UID: \"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7\") " pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" Jun 15 03:12:01.430925 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.430900 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntmxt\" (UniqueName: \"kubernetes.io/projected/06bfd370-abe8-4b3a-a82c-41e7d0ade9c7-kube-api-access-ntmxt\") pod \"insights-operator-5bbd86d6bd-dswhg\" (UID: \"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7\") " pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" Jun 15 03:12:01.561993 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.561885 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" Jun 15 03:12:01.687604 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:01.687574 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5bbd86d6bd-dswhg"] Jun 15 03:12:01.690288 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:12:01.690259 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06bfd370_abe8_4b3a_a82c_41e7d0ade9c7.slice/crio-1768e284dfab2c2e75a1de2b7e72eac173dd270065c6bc07da64bcc0997f9cd8 WatchSource:0}: Error finding container 1768e284dfab2c2e75a1de2b7e72eac173dd270065c6bc07da64bcc0997f9cd8: Status 404 returned error can't find the container with id 1768e284dfab2c2e75a1de2b7e72eac173dd270065c6bc07da64bcc0997f9cd8 Jun 15 03:12:02.663767 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:02.663697 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" event={"ID":"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7","Type":"ContainerStarted","Data":"1768e284dfab2c2e75a1de2b7e72eac173dd270065c6bc07da64bcc0997f9cd8"} Jun 15 03:12:04.671502 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:04.671463 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" event={"ID":"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7","Type":"ContainerStarted","Data":"cf94dd2b2f8707c9e4eefcc5ef96db8c54b381523f9aebb96f1c64757dabe606"} Jun 15 03:12:04.687312 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:04.687256 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" podStartSLOduration=1.4657329909999999 podStartE2EDuration="3.687238006s" totalImagesPullingTime="2.221505015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:12:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:04.686231699 +0000 UTC m=+192.997370689" watchObservedRunningTime="2026-06-15 03:12:04.687238006 +0000 UTC m=+192.998376999" Jun 15 03:12:05.626722 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:05.626689 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pmfbb"] Jun 15 03:12:05.629915 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:05.629887 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:05.631939 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:05.631914 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 15 03:12:05.632065 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:05.631914 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nvxls\"" Jun 15 03:12:05.632651 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:05.632630 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 15 03:12:05.638815 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:05.638787 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pmfbb"] Jun 15 03:12:05.755572 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:05.755539 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-data-volume\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:05.756083 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:05.755624 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:05.756083 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:05.755693 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-crio-socket\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:05.756083 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:05.755767 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:05.756083 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:05.755796 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhv62\" (UniqueName: \"kubernetes.io/projected/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-api-access-vhv62\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:05.856986 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:05.856953 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:05.856986 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:05.856987 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhv62\" (UniqueName: \"kubernetes.io/projected/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-api-access-vhv62\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:05.857189 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:05.857011 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-data-volume\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:05.857189 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:05.857075 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 03:12:05.857189 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:05.857159 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-rbac-proxy-cm podName:b0fe013e-5d5e-441f-bda2-50fc7a5c2140 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:06.357142779 +0000 UTC m=+194.668281760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pmfbb" (UID: "b0fe013e-5d5e-441f-bda2-50fc7a5c2140") : configmap "kube-rbac-proxy" not found Jun 15 03:12:05.857291 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:05.857212 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:05.857291 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:05.857258 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-crio-socket\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:05.857291 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:05.857279 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-data-volume\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:05.857399 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:05.857335 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-crio-socket\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:05.857399 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:05.857345 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 03:12:05.857399 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:05.857393 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls podName:b0fe013e-5d5e-441f-bda2-50fc7a5c2140 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:06.357381718 +0000 UTC m=+194.668520686 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pmfbb" (UID: "b0fe013e-5d5e-441f-bda2-50fc7a5c2140") : secret "insights-runtime-extractor-tls" not found Jun 15 03:12:05.866898 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:05.866870 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhv62\" (UniqueName: \"kubernetes.io/projected/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-api-access-vhv62\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:06.361620 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:06.361588 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:06.361819 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:06.361647 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:06.361819 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:06.361756 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 03:12:06.361961 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:06.361765 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 03:12:06.361961 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:06.361854 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls podName:b0fe013e-5d5e-441f-bda2-50fc7a5c2140 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:07.361812173 +0000 UTC m=+195.672951155 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pmfbb" (UID: "b0fe013e-5d5e-441f-bda2-50fc7a5c2140") : secret "insights-runtime-extractor-tls" not found Jun 15 03:12:06.361961 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:06.361897 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-rbac-proxy-cm podName:b0fe013e-5d5e-441f-bda2-50fc7a5c2140 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:07.361885735 +0000 UTC m=+195.673024703 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pmfbb" (UID: "b0fe013e-5d5e-441f-bda2-50fc7a5c2140") : configmap "kube-rbac-proxy" not found Jun 15 03:12:07.368596 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:07.368534 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:07.368596 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:07.368608 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:07.369119 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:07.368696 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 03:12:07.369119 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:07.368727 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 03:12:07.369119 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:07.368757 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls podName:b0fe013e-5d5e-441f-bda2-50fc7a5c2140 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:09.368743057 +0000 UTC m=+197.679882029 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pmfbb" (UID: "b0fe013e-5d5e-441f-bda2-50fc7a5c2140") : secret "insights-runtime-extractor-tls" not found Jun 15 03:12:07.369119 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:07.368781 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-rbac-proxy-cm podName:b0fe013e-5d5e-441f-bda2-50fc7a5c2140 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:09.368766916 +0000 UTC m=+197.679905888 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pmfbb" (UID: "b0fe013e-5d5e-441f-bda2-50fc7a5c2140") : configmap "kube-rbac-proxy" not found Jun 15 03:12:07.502422 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:07.502394 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lj57j_ff9255c7-dc08-4be9-9087-b749284c6358/dns-node-resolver/0.log" Jun 15 03:12:08.348123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:08.348088 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-tkzgq"] Jun 15 03:12:08.351014 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:08.350998 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7b994bc59-tkzgq" Jun 15 03:12:08.353114 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:08.353088 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-99nzb\"" Jun 15 03:12:08.353227 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:08.353136 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 15 03:12:08.353227 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:08.353136 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 15 03:12:08.353747 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:08.353734 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 15 03:12:08.353804 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:08.353790 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 15 03:12:08.359548 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:08.359529 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-tkzgq"] Jun 15 03:12:08.476137 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:08.476108 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-576qs\" (UniqueName: \"kubernetes.io/projected/a8468ef8-b1f8-4c18-8e7f-5145ac1c6cb9-kube-api-access-576qs\") pod \"service-ca-7b994bc59-tkzgq\" (UID: \"a8468ef8-b1f8-4c18-8e7f-5145ac1c6cb9\") " pod="openshift-service-ca/service-ca-7b994bc59-tkzgq" Jun 15 03:12:08.476515 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:08.476149 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a8468ef8-b1f8-4c18-8e7f-5145ac1c6cb9-signing-cabundle\") pod \"service-ca-7b994bc59-tkzgq\" (UID: \"a8468ef8-b1f8-4c18-8e7f-5145ac1c6cb9\") " pod="openshift-service-ca/service-ca-7b994bc59-tkzgq" Jun 15 03:12:08.476515 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:08.476230 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a8468ef8-b1f8-4c18-8e7f-5145ac1c6cb9-signing-key\") pod \"service-ca-7b994bc59-tkzgq\" (UID: \"a8468ef8-b1f8-4c18-8e7f-5145ac1c6cb9\") " pod="openshift-service-ca/service-ca-7b994bc59-tkzgq" Jun 15 03:12:08.576906 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:08.576868 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a8468ef8-b1f8-4c18-8e7f-5145ac1c6cb9-signing-key\") pod \"service-ca-7b994bc59-tkzgq\" (UID: \"a8468ef8-b1f8-4c18-8e7f-5145ac1c6cb9\") " pod="openshift-service-ca/service-ca-7b994bc59-tkzgq" Jun 15 03:12:08.577074 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:08.576934 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-576qs\" (UniqueName: \"kubernetes.io/projected/a8468ef8-b1f8-4c18-8e7f-5145ac1c6cb9-kube-api-access-576qs\") pod \"service-ca-7b994bc59-tkzgq\" (UID: \"a8468ef8-b1f8-4c18-8e7f-5145ac1c6cb9\") " pod="openshift-service-ca/service-ca-7b994bc59-tkzgq" Jun 15 03:12:08.577074 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:08.576959 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a8468ef8-b1f8-4c18-8e7f-5145ac1c6cb9-signing-cabundle\") pod \"service-ca-7b994bc59-tkzgq\" (UID: \"a8468ef8-b1f8-4c18-8e7f-5145ac1c6cb9\") " pod="openshift-service-ca/service-ca-7b994bc59-tkzgq" Jun 15 03:12:08.577539 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:08.577522 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a8468ef8-b1f8-4c18-8e7f-5145ac1c6cb9-signing-cabundle\") pod \"service-ca-7b994bc59-tkzgq\" (UID: \"a8468ef8-b1f8-4c18-8e7f-5145ac1c6cb9\") " pod="openshift-service-ca/service-ca-7b994bc59-tkzgq" Jun 15 03:12:08.579386 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:08.579366 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a8468ef8-b1f8-4c18-8e7f-5145ac1c6cb9-signing-key\") pod \"service-ca-7b994bc59-tkzgq\" (UID: \"a8468ef8-b1f8-4c18-8e7f-5145ac1c6cb9\") " pod="openshift-service-ca/service-ca-7b994bc59-tkzgq" Jun 15 03:12:08.585107 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:08.585088 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-576qs\" (UniqueName: \"kubernetes.io/projected/a8468ef8-b1f8-4c18-8e7f-5145ac1c6cb9-kube-api-access-576qs\") pod \"service-ca-7b994bc59-tkzgq\" (UID: \"a8468ef8-b1f8-4c18-8e7f-5145ac1c6cb9\") " pod="openshift-service-ca/service-ca-7b994bc59-tkzgq" Jun 15 03:12:08.660210 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:08.660170 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7b994bc59-tkzgq" Jun 15 03:12:08.782050 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:08.780442 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-tkzgq"] Jun 15 03:12:08.783903 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:12:08.783856 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8468ef8_b1f8_4c18_8e7f_5145ac1c6cb9.slice/crio-69a4c2fb841fd19d071f03b73423acb2e868d1d61dcd1e57be10d887c7bf4804 WatchSource:0}: Error finding container 69a4c2fb841fd19d071f03b73423acb2e868d1d61dcd1e57be10d887c7bf4804: Status 404 returned error can't find the container with id 69a4c2fb841fd19d071f03b73423acb2e868d1d61dcd1e57be10d887c7bf4804 Jun 15 03:12:08.902532 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:08.902505 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xhccz_7fe3e8d4-b5da-4511-a7f8-f28596c5f336/node-ca/0.log" Jun 15 03:12:09.384015 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:09.383973 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:09.384197 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:09.384053 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:09.384197 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:09.384138 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 03:12:09.384197 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:09.384171 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 03:12:09.384366 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:09.384221 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls podName:b0fe013e-5d5e-441f-bda2-50fc7a5c2140 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:13.3841986 +0000 UTC m=+201.695337589 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pmfbb" (UID: "b0fe013e-5d5e-441f-bda2-50fc7a5c2140") : secret "insights-runtime-extractor-tls" not found Jun 15 03:12:09.384366 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:09.384241 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-rbac-proxy-cm podName:b0fe013e-5d5e-441f-bda2-50fc7a5c2140 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:13.38423134 +0000 UTC m=+201.695370312 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pmfbb" (UID: "b0fe013e-5d5e-441f-bda2-50fc7a5c2140") : configmap "kube-rbac-proxy" not found Jun 15 03:12:09.503669 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:09.503591 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p9h4n_f61b9993-27b1-4a1d-b7bc-632802832528/serve-healthcheck-canary/0.log" Jun 15 03:12:09.687322 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:09.687238 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7b994bc59-tkzgq" event={"ID":"a8468ef8-b1f8-4c18-8e7f-5145ac1c6cb9","Type":"ContainerStarted","Data":"69a4c2fb841fd19d071f03b73423acb2e868d1d61dcd1e57be10d887c7bf4804"} Jun 15 03:12:11.694922 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:11.694884 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7b994bc59-tkzgq" event={"ID":"a8468ef8-b1f8-4c18-8e7f-5145ac1c6cb9","Type":"ContainerStarted","Data":"c9beef100bd051717b46c4778726fe696cdb8081e256a06f40dd1676847acb36"} Jun 15 03:12:11.721368 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:11.721303 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7b994bc59-tkzgq" podStartSLOduration=1.778344742 podStartE2EDuration="3.721280526s" totalImagesPullingTime="1.942935784s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:11.719995661 +0000 UTC m=+200.031134651" watchObservedRunningTime="2026-06-15 03:12:11.721280526 +0000 UTC m=+200.032419517" Jun 15 03:12:13.417310 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:13.417269 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:13.417720 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:13.417326 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:13.417720 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:13.417390 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 15 03:12:13.417720 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:13.417421 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 03:12:13.417720 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:13.417446 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-rbac-proxy-cm podName:b0fe013e-5d5e-441f-bda2-50fc7a5c2140 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:21.417431126 +0000 UTC m=+209.728570096 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pmfbb" (UID: "b0fe013e-5d5e-441f-bda2-50fc7a5c2140") : configmap "kube-rbac-proxy" not found Jun 15 03:12:13.417720 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:13.417481 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls podName:b0fe013e-5d5e-441f-bda2-50fc7a5c2140 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:21.417467911 +0000 UTC m=+209.728606879 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pmfbb" (UID: "b0fe013e-5d5e-441f-bda2-50fc7a5c2140") : secret "insights-runtime-extractor-tls" not found Jun 15 03:12:21.484628 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:21.484589 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:21.485131 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:21.484645 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:21.485131 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:21.484767 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 15 03:12:21.485131 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:21.484875 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls podName:b0fe013e-5d5e-441f-bda2-50fc7a5c2140 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:37.48485382 +0000 UTC m=+225.795992805 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pmfbb" (UID: "b0fe013e-5d5e-441f-bda2-50fc7a5c2140") : secret "insights-runtime-extractor-tls" not found Jun 15 03:12:21.485285 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:21.485150 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:21.914728 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:21.914687 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" podUID="193678ba-2d80-4a74-a734-4cbc938e653f" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 15 03:12:28.628491 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:28.628461 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-694c955b9-xqnxf"] Jun 15 03:12:28.632469 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:28.632444 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-694c955b9-xqnxf" Jun 15 03:12:28.634524 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:28.634505 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 15 03:12:28.634803 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:28.634770 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 15 03:12:28.634969 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:28.634953 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-mhvss\"" Jun 15 03:12:28.640981 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:28.640962 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-694c955b9-xqnxf"] Jun 15 03:12:28.740672 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:28.740640 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhbpq\" (UniqueName: \"kubernetes.io/projected/3d282f55-36e3-45f9-9a11-505ef16fc6c6-kube-api-access-mhbpq\") pod \"downloads-694c955b9-xqnxf\" (UID: \"3d282f55-36e3-45f9-9a11-505ef16fc6c6\") " pod="openshift-console/downloads-694c955b9-xqnxf" Jun 15 03:12:28.841617 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:28.841585 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhbpq\" (UniqueName: \"kubernetes.io/projected/3d282f55-36e3-45f9-9a11-505ef16fc6c6-kube-api-access-mhbpq\") pod \"downloads-694c955b9-xqnxf\" (UID: \"3d282f55-36e3-45f9-9a11-505ef16fc6c6\") " pod="openshift-console/downloads-694c955b9-xqnxf" Jun 15 03:12:28.851801 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:28.851773 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhbpq\" (UniqueName: \"kubernetes.io/projected/3d282f55-36e3-45f9-9a11-505ef16fc6c6-kube-api-access-mhbpq\") pod \"downloads-694c955b9-xqnxf\" (UID: \"3d282f55-36e3-45f9-9a11-505ef16fc6c6\") " pod="openshift-console/downloads-694c955b9-xqnxf" Jun 15 03:12:28.941988 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:28.941894 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-694c955b9-xqnxf" Jun 15 03:12:29.062544 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:29.062512 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-694c955b9-xqnxf"] Jun 15 03:12:29.067526 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:12:29.067499 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d282f55_36e3_45f9_9a11_505ef16fc6c6.slice/crio-b86805b745d21285b9d4f6b89da611f67dbdddf4c7227dfe40d29f4971265cf3 WatchSource:0}: Error finding container b86805b745d21285b9d4f6b89da611f67dbdddf4c7227dfe40d29f4971265cf3: Status 404 returned error can't find the container with id b86805b745d21285b9d4f6b89da611f67dbdddf4c7227dfe40d29f4971265cf3 Jun 15 03:12:29.747511 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:29.747477 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-694c955b9-xqnxf" event={"ID":"3d282f55-36e3-45f9-9a11-505ef16fc6c6","Type":"ContainerStarted","Data":"b86805b745d21285b9d4f6b89da611f67dbdddf4c7227dfe40d29f4971265cf3"} Jun 15 03:12:31.913989 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:31.913944 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" podUID="193678ba-2d80-4a74-a734-4cbc938e653f" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 15 03:12:37.510145 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:37.510109 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:37.512657 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:37.512636 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b0fe013e-5d5e-441f-bda2-50fc7a5c2140-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmfbb\" (UID: \"b0fe013e-5d5e-441f-bda2-50fc7a5c2140\") " pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:37.740490 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:37.740446 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pmfbb" Jun 15 03:12:37.883440 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:37.883406 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pmfbb"] Jun 15 03:12:37.887156 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:12:37.887124 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0fe013e_5d5e_441f_bda2_50fc7a5c2140.slice/crio-e9bde9fe5b31c23b4e7c27d16f8aad0de6ef366688d11af1409611adfd01d9a3 WatchSource:0}: Error finding container e9bde9fe5b31c23b4e7c27d16f8aad0de6ef366688d11af1409611adfd01d9a3: Status 404 returned error can't find the container with id e9bde9fe5b31c23b4e7c27d16f8aad0de6ef366688d11af1409611adfd01d9a3 Jun 15 03:12:38.778692 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:38.778612 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pmfbb" event={"ID":"b0fe013e-5d5e-441f-bda2-50fc7a5c2140","Type":"ContainerStarted","Data":"e9bde9fe5b31c23b4e7c27d16f8aad0de6ef366688d11af1409611adfd01d9a3"} Jun 15 03:12:41.914624 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:41.914576 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" podUID="193678ba-2d80-4a74-a734-4cbc938e653f" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 15 03:12:41.915165 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:41.914674 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" Jun 15 03:12:41.915270 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:41.915239 2562 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"99a9fb0721d26f993721030cdaa2cbc2e43933bb2c715a8ad73a4bbba0a69566"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jun 15 03:12:41.915312 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:41.915295 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" podUID="193678ba-2d80-4a74-a734-4cbc938e653f" containerName="service-proxy" containerID="cri-o://99a9fb0721d26f993721030cdaa2cbc2e43933bb2c715a8ad73a4bbba0a69566" gracePeriod=30 Jun 15 03:12:46.068401 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.068365 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx"] Jun 15 03:12:46.074031 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.074009 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx" Jun 15 03:12:46.076566 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.076327 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 15 03:12:46.076799 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.076777 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-lpq6f\"" Jun 15 03:12:46.077200 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.077182 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 15 03:12:46.078231 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.077746 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 15 03:12:46.078231 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.077970 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 15 03:12:46.078231 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.077977 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 15 03:12:46.083890 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.082392 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx"] Jun 15 03:12:46.099904 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.099793 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hmqkw"] Jun 15 03:12:46.104790 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.104730 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.107335 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.106878 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 15 03:12:46.107335 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.106913 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-przxp\"" Jun 15 03:12:46.107335 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.106946 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 15 03:12:46.107335 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.107186 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 15 03:12:46.180899 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.180865 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63801ac3-c391-4e2f-93ca-8f0a495d672d-metrics-client-ca\") pod \"openshift-state-metrics-6b8d965dd5-tnhbx\" (UID: \"63801ac3-c391-4e2f-93ca-8f0a495d672d\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx" Jun 15 03:12:46.180899 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.180911 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/423a0d92-fe2b-4a84-b581-1bec5edad495-node-exporter-tls\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.181133 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.180952 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/423a0d92-fe2b-4a84-b581-1bec5edad495-root\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.181133 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.180994 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/423a0d92-fe2b-4a84-b581-1bec5edad495-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.181133 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.181044 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27glj\" (UniqueName: \"kubernetes.io/projected/423a0d92-fe2b-4a84-b581-1bec5edad495-kube-api-access-27glj\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.181133 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.181071 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63801ac3-c391-4e2f-93ca-8f0a495d672d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6b8d965dd5-tnhbx\" (UID: \"63801ac3-c391-4e2f-93ca-8f0a495d672d\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx" Jun 15 03:12:46.181133 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.181088 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/423a0d92-fe2b-4a84-b581-1bec5edad495-node-exporter-accelerators-collector-config\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.181133 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.181116 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/423a0d92-fe2b-4a84-b581-1bec5edad495-node-exporter-textfile\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.181430 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.181171 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/423a0d92-fe2b-4a84-b581-1bec5edad495-sys\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.181430 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.181193 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78v2f\" (UniqueName: \"kubernetes.io/projected/63801ac3-c391-4e2f-93ca-8f0a495d672d-kube-api-access-78v2f\") pod \"openshift-state-metrics-6b8d965dd5-tnhbx\" (UID: \"63801ac3-c391-4e2f-93ca-8f0a495d672d\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx" Jun 15 03:12:46.181430 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.181221 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/63801ac3-c391-4e2f-93ca-8f0a495d672d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6b8d965dd5-tnhbx\" (UID: \"63801ac3-c391-4e2f-93ca-8f0a495d672d\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx" Jun 15 03:12:46.181430 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.181263 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/423a0d92-fe2b-4a84-b581-1bec5edad495-node-exporter-wtmp\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.181430 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.181294 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/423a0d92-fe2b-4a84-b581-1bec5edad495-metrics-client-ca\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.283191 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.283120 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/423a0d92-fe2b-4a84-b581-1bec5edad495-node-exporter-accelerators-collector-config\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.283191 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.283161 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/423a0d92-fe2b-4a84-b581-1bec5edad495-node-exporter-textfile\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.283406 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.283211 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/423a0d92-fe2b-4a84-b581-1bec5edad495-sys\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.283406 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.283238 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78v2f\" (UniqueName: \"kubernetes.io/projected/63801ac3-c391-4e2f-93ca-8f0a495d672d-kube-api-access-78v2f\") pod \"openshift-state-metrics-6b8d965dd5-tnhbx\" (UID: \"63801ac3-c391-4e2f-93ca-8f0a495d672d\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx" Jun 15 03:12:46.283406 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.283266 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/63801ac3-c391-4e2f-93ca-8f0a495d672d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6b8d965dd5-tnhbx\" (UID: \"63801ac3-c391-4e2f-93ca-8f0a495d672d\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx" Jun 15 03:12:46.283406 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.283305 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/423a0d92-fe2b-4a84-b581-1bec5edad495-node-exporter-wtmp\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.283406 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.283337 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/423a0d92-fe2b-4a84-b581-1bec5edad495-metrics-client-ca\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.283406 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.283370 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63801ac3-c391-4e2f-93ca-8f0a495d672d-metrics-client-ca\") pod \"openshift-state-metrics-6b8d965dd5-tnhbx\" (UID: \"63801ac3-c391-4e2f-93ca-8f0a495d672d\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx" Jun 15 03:12:46.283705 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.283416 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/423a0d92-fe2b-4a84-b581-1bec5edad495-node-exporter-tls\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.283705 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.283455 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/423a0d92-fe2b-4a84-b581-1bec5edad495-root\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.283705 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.283480 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/423a0d92-fe2b-4a84-b581-1bec5edad495-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.283705 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.283505 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27glj\" (UniqueName: \"kubernetes.io/projected/423a0d92-fe2b-4a84-b581-1bec5edad495-kube-api-access-27glj\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.283705 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.283532 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63801ac3-c391-4e2f-93ca-8f0a495d672d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6b8d965dd5-tnhbx\" (UID: \"63801ac3-c391-4e2f-93ca-8f0a495d672d\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx" Jun 15 03:12:46.284021 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.283738 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/423a0d92-fe2b-4a84-b581-1bec5edad495-node-exporter-accelerators-collector-config\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.284021 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.283985 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/423a0d92-fe2b-4a84-b581-1bec5edad495-root\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.285301 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.284181 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/423a0d92-fe2b-4a84-b581-1bec5edad495-metrics-client-ca\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.285301 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.284455 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/423a0d92-fe2b-4a84-b581-1bec5edad495-node-exporter-textfile\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.285301 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.284513 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/423a0d92-fe2b-4a84-b581-1bec5edad495-sys\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.285301 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.284589 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63801ac3-c391-4e2f-93ca-8f0a495d672d-metrics-client-ca\") pod \"openshift-state-metrics-6b8d965dd5-tnhbx\" (UID: \"63801ac3-c391-4e2f-93ca-8f0a495d672d\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx" Jun 15 03:12:46.285301 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:46.284659 2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 15 03:12:46.285301 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:46.284715 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/423a0d92-fe2b-4a84-b581-1bec5edad495-node-exporter-tls podName:423a0d92-fe2b-4a84-b581-1bec5edad495 nodeName:}" failed. No retries permitted until 2026-06-15 03:12:46.784698896 +0000 UTC m=+235.095837874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/423a0d92-fe2b-4a84-b581-1bec5edad495-node-exporter-tls") pod "node-exporter-hmqkw" (UID: "423a0d92-fe2b-4a84-b581-1bec5edad495") : secret "node-exporter-tls" not found Jun 15 03:12:46.285301 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:46.284870 2562 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jun 15 03:12:46.285301 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:46.284915 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63801ac3-c391-4e2f-93ca-8f0a495d672d-openshift-state-metrics-tls podName:63801ac3-c391-4e2f-93ca-8f0a495d672d nodeName:}" failed. No retries permitted until 2026-06-15 03:12:46.784901486 +0000 UTC m=+235.096040455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/63801ac3-c391-4e2f-93ca-8f0a495d672d-openshift-state-metrics-tls") pod "openshift-state-metrics-6b8d965dd5-tnhbx" (UID: "63801ac3-c391-4e2f-93ca-8f0a495d672d") : secret "openshift-state-metrics-tls" not found Jun 15 03:12:46.285301 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.285038 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/423a0d92-fe2b-4a84-b581-1bec5edad495-node-exporter-wtmp\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.288497 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.287818 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63801ac3-c391-4e2f-93ca-8f0a495d672d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6b8d965dd5-tnhbx\" (UID: \"63801ac3-c391-4e2f-93ca-8f0a495d672d\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx" Jun 15 03:12:46.288497 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.288100 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/423a0d92-fe2b-4a84-b581-1bec5edad495-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.297490 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.297465 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78v2f\" (UniqueName: \"kubernetes.io/projected/63801ac3-c391-4e2f-93ca-8f0a495d672d-kube-api-access-78v2f\") pod \"openshift-state-metrics-6b8d965dd5-tnhbx\" (UID: \"63801ac3-c391-4e2f-93ca-8f0a495d672d\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx" Jun 15 03:12:46.298215 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.298189 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27glj\" (UniqueName: \"kubernetes.io/projected/423a0d92-fe2b-4a84-b581-1bec5edad495-kube-api-access-27glj\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.788470 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.788414 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/423a0d92-fe2b-4a84-b581-1bec5edad495-node-exporter-tls\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.788647 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.788539 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/63801ac3-c391-4e2f-93ca-8f0a495d672d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6b8d965dd5-tnhbx\" (UID: \"63801ac3-c391-4e2f-93ca-8f0a495d672d\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx" Jun 15 03:12:46.791323 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.791298 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/423a0d92-fe2b-4a84-b581-1bec5edad495-node-exporter-tls\") pod \"node-exporter-hmqkw\" (UID: \"423a0d92-fe2b-4a84-b581-1bec5edad495\") " pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:46.791449 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.791421 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/63801ac3-c391-4e2f-93ca-8f0a495d672d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6b8d965dd5-tnhbx\" (UID: \"63801ac3-c391-4e2f-93ca-8f0a495d672d\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx" Jun 15 03:12:46.808346 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.808309 2562 generic.go:356] "Generic (PLEG): container finished" podID="193678ba-2d80-4a74-a734-4cbc938e653f" containerID="99a9fb0721d26f993721030cdaa2cbc2e43933bb2c715a8ad73a4bbba0a69566" exitCode=2 Jun 15 03:12:46.808346 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.808328 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" event={"ID":"193678ba-2d80-4a74-a734-4cbc938e653f","Type":"ContainerDied","Data":"99a9fb0721d26f993721030cdaa2cbc2e43933bb2c715a8ad73a4bbba0a69566"} Jun 15 03:12:46.987048 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:46.987010 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx" Jun 15 03:12:47.016684 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.016651 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hmqkw" Jun 15 03:12:47.150343 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.150309 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 15 03:12:47.159550 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.159528 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.161632 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.161609 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 15 03:12:47.161775 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.161611 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 15 03:12:47.162297 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.162279 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 15 03:12:47.162420 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.162287 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 15 03:12:47.162538 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.162516 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 15 03:12:47.162845 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.162798 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 15 03:12:47.163026 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.163008 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 15 03:12:47.163026 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.163021 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 15 03:12:47.163170 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.163005 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 15 03:12:47.163252 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.163187 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2vrpx\"" Jun 15 03:12:47.170691 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.170663 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 15 03:12:47.191193 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.191162 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b504416-ace4-481d-97c4-abe48c82896d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.191341 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.191210 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-web-config\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.191341 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.191278 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.191341 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.191320 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4b504416-ace4-481d-97c4-abe48c82896d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.191490 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.191364 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4b504416-ace4-481d-97c4-abe48c82896d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.191490 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.191398 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4b504416-ace4-481d-97c4-abe48c82896d-config-out\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.191490 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.191430 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.191490 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.191458 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.191644 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.191517 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b504416-ace4-481d-97c4-abe48c82896d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.191644 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.191542 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4snq\" (UniqueName: \"kubernetes.io/projected/4b504416-ace4-481d-97c4-abe48c82896d-kube-api-access-b4snq\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.191644 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.191593 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.191644 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.191641 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-config-volume\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.191817 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.191657 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.292453 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.292420 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.292628 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.292483 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-config-volume\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.292685 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.292648 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.292742 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.292692 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b504416-ace4-481d-97c4-abe48c82896d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.292742 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.292727 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-web-config\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.292872 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.292769 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.292872 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.292802 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4b504416-ace4-481d-97c4-abe48c82896d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.292872 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.292848 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4b504416-ace4-481d-97c4-abe48c82896d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.293040 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.292876 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4b504416-ace4-481d-97c4-abe48c82896d-config-out\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.293040 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.292907 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.293040 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.292937 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.293040 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.292985 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b504416-ace4-481d-97c4-abe48c82896d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.293040 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.293010 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4snq\" (UniqueName: \"kubernetes.io/projected/4b504416-ace4-481d-97c4-abe48c82896d-kube-api-access-b4snq\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.293572 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.293537 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b504416-ace4-481d-97c4-abe48c82896d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.295918 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.295571 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.295918 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.295577 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.295918 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.295809 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4b504416-ace4-481d-97c4-abe48c82896d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.296120 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:12:47.295970 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b504416-ace4-481d-97c4-abe48c82896d-alertmanager-trusted-ca-bundle podName:4b504416-ace4-481d-97c4-abe48c82896d nodeName:}" failed. No retries permitted until 2026-06-15 03:12:47.795949419 +0000 UTC m=+236.107088386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/4b504416-ace4-481d-97c4-abe48c82896d-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "4b504416-ace4-481d-97c4-abe48c82896d") : configmap references non-existent config key: ca-bundle.crt Jun 15 03:12:47.296120 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.296080 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.296245 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.296233 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4b504416-ace4-481d-97c4-abe48c82896d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.296417 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.296395 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-web-config\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.296517 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.296449 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-config-volume\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.297022 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.296997 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.297487 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.297469 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4b504416-ace4-481d-97c4-abe48c82896d-config-out\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.298013 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.297998 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.300699 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.300679 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4snq\" (UniqueName: \"kubernetes.io/projected/4b504416-ace4-481d-97c4-abe48c82896d-kube-api-access-b4snq\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.797034 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.796995 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b504416-ace4-481d-97c4-abe48c82896d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:47.797839 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:47.797810 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b504416-ace4-481d-97c4-abe48c82896d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:48.035932 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:12:48.035901 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod423a0d92_fe2b_4a84_b581_1bec5edad495.slice/crio-7d0dce47fdab7c08a71d14da5698d3b00696341b91e86c4a78e1b58e784680d1 WatchSource:0}: Error finding container 7d0dce47fdab7c08a71d14da5698d3b00696341b91e86c4a78e1b58e784680d1: Status 404 returned error can't find the container with id 7d0dce47fdab7c08a71d14da5698d3b00696341b91e86c4a78e1b58e784680d1 Jun 15 03:12:48.071483 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:48.071292 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:12:48.189308 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:48.189192 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx"] Jun 15 03:12:48.194284 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:12:48.194252 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63801ac3_c391_4e2f_93ca_8f0a495d672d.slice/crio-56d83109f366953cba2dd5a3018dc7f331b988dfd3a820a3d06d58217b8f3645 WatchSource:0}: Error finding container 56d83109f366953cba2dd5a3018dc7f331b988dfd3a820a3d06d58217b8f3645: Status 404 returned error can't find the container with id 56d83109f366953cba2dd5a3018dc7f331b988dfd3a820a3d06d58217b8f3645 Jun 15 03:12:48.245374 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:48.245345 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 15 03:12:48.248620 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:12:48.248599 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b504416_ace4_481d_97c4_abe48c82896d.slice/crio-c6479fc4b075daafce820f0043c3f12754bff1e1e1ecd632a514310bc46d8f9a WatchSource:0}: Error finding container c6479fc4b075daafce820f0043c3f12754bff1e1e1ecd632a514310bc46d8f9a: Status 404 returned error can't find the container with id c6479fc4b075daafce820f0043c3f12754bff1e1e1ecd632a514310bc46d8f9a Jun 15 03:12:48.820481 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:48.820441 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76c56fdcc-xtl2w" event={"ID":"193678ba-2d80-4a74-a734-4cbc938e653f","Type":"ContainerStarted","Data":"ced0a87ea3e494cbd077b41641a4c8c189cebb6c335eac412d57f25397033e84"} Jun 15 03:12:48.822702 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:48.822668 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx" event={"ID":"63801ac3-c391-4e2f-93ca-8f0a495d672d","Type":"ContainerStarted","Data":"41f1d6973b10e949caf77c3d998b09d9ed0534014fc5127a68d80f7161fa523b"} Jun 15 03:12:48.822867 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:48.822730 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx" event={"ID":"63801ac3-c391-4e2f-93ca-8f0a495d672d","Type":"ContainerStarted","Data":"c23fde3fabcaece02661afee5e0d5a5de56faeeb6a6ab8a7a27841d9f19c8764"} Jun 15 03:12:48.822867 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:48.822760 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx" event={"ID":"63801ac3-c391-4e2f-93ca-8f0a495d672d","Type":"ContainerStarted","Data":"56d83109f366953cba2dd5a3018dc7f331b988dfd3a820a3d06d58217b8f3645"} Jun 15 03:12:48.824509 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:48.824481 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pmfbb" event={"ID":"b0fe013e-5d5e-441f-bda2-50fc7a5c2140","Type":"ContainerStarted","Data":"210a1274b10bcbe7db9e5a97aa08086c08b11ed081442d2f54e1e4b9237496be"} Jun 15 03:12:48.825925 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:48.825900 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b504416-ace4-481d-97c4-abe48c82896d","Type":"ContainerStarted","Data":"c6479fc4b075daafce820f0043c3f12754bff1e1e1ecd632a514310bc46d8f9a"} Jun 15 03:12:48.827453 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:48.827427 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hmqkw" event={"ID":"423a0d92-fe2b-4a84-b581-1bec5edad495","Type":"ContainerStarted","Data":"7d0dce47fdab7c08a71d14da5698d3b00696341b91e86c4a78e1b58e784680d1"} Jun 15 03:12:48.829285 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:48.829259 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-694c955b9-xqnxf" event={"ID":"3d282f55-36e3-45f9-9a11-505ef16fc6c6","Type":"ContainerStarted","Data":"2a15ce37aa11b31a5825f58911fbcad8b7f395822d35fc1a08c0d2e1b9a03e02"} Jun 15 03:12:48.829800 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:48.829746 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-694c955b9-xqnxf" Jun 15 03:12:48.842353 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:48.842327 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-694c955b9-xqnxf" Jun 15 03:12:48.856490 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:48.856435 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-694c955b9-xqnxf" podStartSLOduration=1.8929831560000001 podStartE2EDuration="20.856415445s" totalImagesPullingTime="18.963432289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:48.855457851 +0000 UTC m=+237.166596838" watchObservedRunningTime="2026-06-15 03:12:48.856415445 +0000 UTC m=+237.167554436" Jun 15 03:12:49.836992 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:49.836949 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pmfbb" event={"ID":"b0fe013e-5d5e-441f-bda2-50fc7a5c2140","Type":"ContainerStarted","Data":"1d572ee3d160396e0903af7e0c0ee2039afcebec7fc30e60959d43c88ee91466"} Jun 15 03:12:49.839050 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:49.839016 2562 generic.go:356] "Generic (PLEG): container finished" podID="423a0d92-fe2b-4a84-b581-1bec5edad495" containerID="b871b4208fc52798a1b1d86f90e4fe9f431f7c3a4a51b0a33a4d6114505b810a" exitCode=0 Jun 15 03:12:49.839235 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:49.839200 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hmqkw" event={"ID":"423a0d92-fe2b-4a84-b581-1bec5edad495","Type":"ContainerDied","Data":"b871b4208fc52798a1b1d86f90e4fe9f431f7c3a4a51b0a33a4d6114505b810a"} Jun 15 03:12:50.848353 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:50.848267 2562 generic.go:356] "Generic (PLEG): container finished" podID="4b504416-ace4-481d-97c4-abe48c82896d" containerID="f684b23f9d5cc7d8c4d2e64c95a0410bd50673e53c9c3c800ee2ef51ff17b5d1" exitCode=0 Jun 15 03:12:50.848780 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:50.848366 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b504416-ace4-481d-97c4-abe48c82896d","Type":"ContainerDied","Data":"f684b23f9d5cc7d8c4d2e64c95a0410bd50673e53c9c3c800ee2ef51ff17b5d1"} Jun 15 03:12:50.853105 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:50.853072 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hmqkw" event={"ID":"423a0d92-fe2b-4a84-b581-1bec5edad495","Type":"ContainerStarted","Data":"1e0a607bcbfce201a594d60db5ac3ae4da80035955fc8452cd8c25e6c5a97fdd"} Jun 15 03:12:50.853221 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:50.853125 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hmqkw" event={"ID":"423a0d92-fe2b-4a84-b581-1bec5edad495","Type":"ContainerStarted","Data":"5e0ec05f580307fa97a841c5cddafbef8f56c89ca50529aa281693120ed202e4"} Jun 15 03:12:50.856292 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:50.856154 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx" event={"ID":"63801ac3-c391-4e2f-93ca-8f0a495d672d","Type":"ContainerStarted","Data":"00e9b5cb3cfa175542dafbd02ba9430d7eff8b457d2694c0c46290398029d30f"} Jun 15 03:12:50.861748 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:50.861708 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-589576dfc7-pzl7w"] Jun 15 03:12:50.883023 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:50.882894 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-589576dfc7-pzl7w"] Jun 15 03:12:50.883023 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:50.883027 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-589576dfc7-pzl7w" Jun 15 03:12:50.885156 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:50.885072 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 15 03:12:50.885397 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:50.885376 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-hznpf\"" Jun 15 03:12:50.899560 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:50.899520 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-tnhbx" podStartSLOduration=2.6854869 podStartE2EDuration="4.899505076s" totalImagesPullingTime="2.214018176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:50.898199417 +0000 UTC m=+239.209338406" watchObservedRunningTime="2026-06-15 03:12:50.899505076 +0000 UTC m=+239.210644066" Jun 15 03:12:50.913668 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:50.913617 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hmqkw" podStartSLOduration=3.746304785 podStartE2EDuration="4.913597925s" totalImagesPullingTime="1.056515517s" totalInitContainerRuntime="110.777623ms" isStatefulPod=true podCreationTimestamp="2026-06-15 03:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:50.913144265 +0000 UTC m=+239.224283278" watchObservedRunningTime="2026-06-15 03:12:50.913597925 +0000 UTC m=+239.224736916" Jun 15 03:12:50.927931 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:50.926450 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/455c4a83-df24-49bb-b1a2-f427ed4a795f-monitoring-plugin-cert\") pod \"monitoring-plugin-589576dfc7-pzl7w\" (UID: \"455c4a83-df24-49bb-b1a2-f427ed4a795f\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-pzl7w" Jun 15 03:12:50.927931 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:50.927122 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ds5c\" (UniqueName: \"kubernetes.io/projected/455c4a83-df24-49bb-b1a2-f427ed4a795f-kube-api-access-5ds5c\") pod \"monitoring-plugin-589576dfc7-pzl7w\" (UID: \"455c4a83-df24-49bb-b1a2-f427ed4a795f\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-pzl7w" Jun 15 03:12:51.028296 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:51.028139 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/455c4a83-df24-49bb-b1a2-f427ed4a795f-monitoring-plugin-cert\") pod \"monitoring-plugin-589576dfc7-pzl7w\" (UID: \"455c4a83-df24-49bb-b1a2-f427ed4a795f\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-pzl7w" Jun 15 03:12:51.028296 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:51.028218 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ds5c\" (UniqueName: \"kubernetes.io/projected/455c4a83-df24-49bb-b1a2-f427ed4a795f-kube-api-access-5ds5c\") pod \"monitoring-plugin-589576dfc7-pzl7w\" (UID: \"455c4a83-df24-49bb-b1a2-f427ed4a795f\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-pzl7w" Jun 15 03:12:51.031686 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:51.031658 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/455c4a83-df24-49bb-b1a2-f427ed4a795f-monitoring-plugin-cert\") pod \"monitoring-plugin-589576dfc7-pzl7w\" (UID: \"455c4a83-df24-49bb-b1a2-f427ed4a795f\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-pzl7w" Jun 15 03:12:51.036293 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:51.036218 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ds5c\" (UniqueName: \"kubernetes.io/projected/455c4a83-df24-49bb-b1a2-f427ed4a795f-kube-api-access-5ds5c\") pod \"monitoring-plugin-589576dfc7-pzl7w\" (UID: \"455c4a83-df24-49bb-b1a2-f427ed4a795f\") " pod="openshift-monitoring/monitoring-plugin-589576dfc7-pzl7w" Jun 15 03:12:51.195678 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:51.195638 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-589576dfc7-pzl7w" Jun 15 03:12:52.212450 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.212356 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-589576dfc7-pzl7w"] Jun 15 03:12:52.216555 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:12:52.216525 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod455c4a83_df24_49bb_b1a2_f427ed4a795f.slice/crio-36a5a68bb64bc24261b0c24d591b7a5798f4b808b3da2853d38ff0a89340ab07 WatchSource:0}: Error finding container 36a5a68bb64bc24261b0c24d591b7a5798f4b808b3da2853d38ff0a89340ab07: Status 404 returned error can't find the container with id 36a5a68bb64bc24261b0c24d591b7a5798f4b808b3da2853d38ff0a89340ab07 Jun 15 03:12:52.326401 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.326322 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 15 03:12:52.344947 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.344916 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.347518 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.347488 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-t2cln\"" Jun 15 03:12:52.347702 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.347680 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 15 03:12:52.348061 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.348034 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7ooqann5mhn16\"" Jun 15 03:12:52.348647 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.348622 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 15 03:12:52.349994 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.349977 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 15 03:12:52.350509 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.350283 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 15 03:12:52.350509 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.350358 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 15 03:12:52.350509 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.350383 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 15 03:12:52.350716 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.350566 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 15 03:12:52.351034 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.350873 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 15 03:12:52.351034 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.350929 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 15 03:12:52.351182 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.351078 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 15 03:12:52.351182 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.351179 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 15 03:12:52.351872 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.351852 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 15 03:12:52.352011 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.351882 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 15 03:12:52.353654 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.353631 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 15 03:12:52.354253 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.354231 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 15 03:12:52.356677 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.356656 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 15 03:12:52.443638 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.443600 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-config\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.443820 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.443648 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.443820 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.443736 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/790ce4b0-43e7-48a9-a24b-954ade22f05b-config-out\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.443977 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.443856 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.443977 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.443903 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.443977 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.443931 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.444129 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.443981 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.444129 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.444031 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.444129 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.444089 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.444129 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.444122 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.444305 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.444177 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.444305 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.444211 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kflj6\" (UniqueName: \"kubernetes.io/projected/790ce4b0-43e7-48a9-a24b-954ade22f05b-kube-api-access-kflj6\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.444305 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.444236 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.444305 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.444275 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.444305 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.444295 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.444503 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.444320 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-web-config\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.444503 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.444343 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.444503 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.444367 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.444503 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.444391 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/790ce4b0-43e7-48a9-a24b-954ade22f05b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.444503 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.444417 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.544868 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.544807 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/790ce4b0-43e7-48a9-a24b-954ade22f05b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.545193 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.544878 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.545193 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.544910 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-config\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.545193 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.544941 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.545193 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.545000 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/790ce4b0-43e7-48a9-a24b-954ade22f05b-config-out\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.545193 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.545051 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.545193 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.545080 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.545193 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.545106 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.545193 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.545131 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.545193 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.545160 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.545606 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.545187 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.545606 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.545238 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.545606 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.545278 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.545606 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.545316 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kflj6\" (UniqueName: \"kubernetes.io/projected/790ce4b0-43e7-48a9-a24b-954ade22f05b-kube-api-access-kflj6\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.545606 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.545347 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.545606 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.545374 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.545606 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.545399 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.545606 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.545429 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-web-config\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.545606 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.545457 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.545606 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.545488 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.548572 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.548546 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.549455 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.549430 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.552084 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.552051 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.552193 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.552165 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.552722 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.552699 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.553425 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.553064 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/790ce4b0-43e7-48a9-a24b-954ade22f05b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.553699 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.553676 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.557002 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.556980 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.557082 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.556998 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.557082 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.557031 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.557320 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.557296 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.557854 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.557806 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-config\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.558332 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.557936 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.558332 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.558193 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.559782 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.559735 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.559782 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.559750 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.563102 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.563002 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/790ce4b0-43e7-48a9-a24b-954ade22f05b-config-out\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.567233 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.567208 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.571236 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.571183 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-web-config\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.579726 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.579670 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kflj6\" (UniqueName: \"kubernetes.io/projected/790ce4b0-43e7-48a9-a24b-954ade22f05b-kube-api-access-kflj6\") pod \"prometheus-k8s-0\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.658939 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.658898 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:12:52.850700 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.850300 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 15 03:12:52.852756 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:12:52.852721 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod790ce4b0_43e7_48a9_a24b_954ade22f05b.slice/crio-27d37857ae3d3c397ddeb5916d9aac9a29a828528f72991fe3b69b9e55707e71 WatchSource:0}: Error finding container 27d37857ae3d3c397ddeb5916d9aac9a29a828528f72991fe3b69b9e55707e71: Status 404 returned error can't find the container with id 27d37857ae3d3c397ddeb5916d9aac9a29a828528f72991fe3b69b9e55707e71 Jun 15 03:12:52.868509 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.868478 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-589576dfc7-pzl7w" event={"ID":"455c4a83-df24-49bb-b1a2-f427ed4a795f","Type":"ContainerStarted","Data":"36a5a68bb64bc24261b0c24d591b7a5798f4b808b3da2853d38ff0a89340ab07"} Jun 15 03:12:52.871303 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.871274 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pmfbb" event={"ID":"b0fe013e-5d5e-441f-bda2-50fc7a5c2140","Type":"ContainerStarted","Data":"11772fdcbc5fd8ae67b2134ed54a1dff8814f1fbdf649fc29acce423d7123fca"} Jun 15 03:12:52.873067 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:52.873045 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"790ce4b0-43e7-48a9-a24b-954ade22f05b","Type":"ContainerStarted","Data":"27d37857ae3d3c397ddeb5916d9aac9a29a828528f72991fe3b69b9e55707e71"} Jun 15 03:12:53.879604 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:53.879563 2562 generic.go:356] "Generic (PLEG): container finished" podID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerID="e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7" exitCode=0 Jun 15 03:12:53.880245 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:53.879614 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"790ce4b0-43e7-48a9-a24b-954ade22f05b","Type":"ContainerDied","Data":"e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7"} Jun 15 03:12:53.905722 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:53.905670 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pmfbb" podStartSLOduration=34.530024276 podStartE2EDuration="48.905652782s" totalImagesPullingTime="14.375628506s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:12:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:52.900911465 +0000 UTC m=+241.212050489" watchObservedRunningTime="2026-06-15 03:12:53.905652782 +0000 UTC m=+242.216791772" Jun 15 03:12:55.898537 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:55.898421 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b504416-ace4-481d-97c4-abe48c82896d","Type":"ContainerStarted","Data":"a3dbef18c6c0d37ddefeff2c2dc07af87f921b33c773940b8910ed322c7bcef2"} Jun 15 03:12:55.898537 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:55.898509 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b504416-ace4-481d-97c4-abe48c82896d","Type":"ContainerStarted","Data":"b9e43e8d510f233bd0a7f4895a4abcbe2a4f0b07aa896911a6d1ccee6bb16472"} Jun 15 03:12:55.899138 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:55.898552 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b504416-ace4-481d-97c4-abe48c82896d","Type":"ContainerStarted","Data":"dcd6c540d6e259233c2be7dd63ef6f52c246a5953b89977b1e942366d41db1a7"} Jun 15 03:12:55.899138 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:55.898591 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b504416-ace4-481d-97c4-abe48c82896d","Type":"ContainerStarted","Data":"9a5926518e11f03e1f674754c7d5b569c03a5c1f77e478db0b4ecf4f94a869e7"} Jun 15 03:12:55.899138 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:55.898623 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b504416-ace4-481d-97c4-abe48c82896d","Type":"ContainerStarted","Data":"55502f1b3c2d8ac5da6eb43ead041c70a745ef1b89da32c2e9a1f1d8cd539af8"} Jun 15 03:12:55.901656 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:55.900951 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-589576dfc7-pzl7w" event={"ID":"455c4a83-df24-49bb-b1a2-f427ed4a795f","Type":"ContainerStarted","Data":"40ede6894a539120a2d24d40c4ff3a5429faa401c48a2c8428533f199a0cbb61"} Jun 15 03:12:55.901656 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:55.901503 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-589576dfc7-pzl7w" Jun 15 03:12:55.908095 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:55.908047 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-589576dfc7-pzl7w" Jun 15 03:12:55.933958 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:55.933392 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-589576dfc7-pzl7w" podStartSLOduration=3.332506371 podStartE2EDuration="5.933372064s" totalImagesPullingTime="2.600865693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:12:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:55.917381873 +0000 UTC m=+244.228520875" watchObservedRunningTime="2026-06-15 03:12:55.933372064 +0000 UTC m=+244.244511055" Jun 15 03:12:58.915808 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:58.915771 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b504416-ace4-481d-97c4-abe48c82896d","Type":"ContainerStarted","Data":"22a13ac11431b668e455f1483ad013325d2a1b69c74b515a75bea7e92b83f68a"} Jun 15 03:12:58.917975 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:58.917952 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"790ce4b0-43e7-48a9-a24b-954ade22f05b","Type":"ContainerStarted","Data":"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e"} Jun 15 03:12:58.918088 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:58.918006 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"790ce4b0-43e7-48a9-a24b-954ade22f05b","Type":"ContainerStarted","Data":"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960"} Jun 15 03:12:58.947187 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:12:58.947136 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.097083661 podStartE2EDuration="11.947114838s" totalImagesPullingTime="8.842124717s" totalInitContainerRuntime="7.90646ms" isStatefulPod=false podCreationTimestamp="2026-06-15 03:12:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:12:58.945451169 +0000 UTC m=+247.256590171" watchObservedRunningTime="2026-06-15 03:12:58.947114838 +0000 UTC m=+247.258253829" Jun 15 03:13:01.932728 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:01.932695 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"790ce4b0-43e7-48a9-a24b-954ade22f05b","Type":"ContainerStarted","Data":"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e"} Jun 15 03:13:01.933196 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:01.932775 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"790ce4b0-43e7-48a9-a24b-954ade22f05b","Type":"ContainerStarted","Data":"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a"} Jun 15 03:13:01.933196 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:01.932814 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"790ce4b0-43e7-48a9-a24b-954ade22f05b","Type":"ContainerStarted","Data":"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8"} Jun 15 03:13:01.933196 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:01.932878 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"790ce4b0-43e7-48a9-a24b-954ade22f05b","Type":"ContainerStarted","Data":"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae"} Jun 15 03:13:01.962069 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:01.962009 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.028778871 podStartE2EDuration="9.961988497s" totalImagesPullingTime="6.905740531s" totalInitContainerRuntime="27.469095ms" isStatefulPod=false podCreationTimestamp="2026-06-15 03:12:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:13:01.960884304 +0000 UTC m=+250.272023293" watchObservedRunningTime="2026-06-15 03:13:01.961988497 +0000 UTC m=+250.273127487" Jun 15 03:13:02.659792 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:02.659756 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:13:03.858541 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:03.858498 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs\") pod \"network-metrics-daemon-vz9g7\" (UID: \"c85b7d1f-4d4b-4fdd-9427-d1c25942aebe\") " pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:13:03.860784 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:03.860761 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85b7d1f-4d4b-4fdd-9427-d1c25942aebe-metrics-certs\") pod \"network-metrics-daemon-vz9g7\" (UID: \"c85b7d1f-4d4b-4fdd-9427-d1c25942aebe\") " pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:13:04.043475 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:04.043429 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lnpms\"" Jun 15 03:13:04.051968 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:04.051939 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vz9g7" Jun 15 03:13:04.203237 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:04.203201 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vz9g7"] Jun 15 03:13:04.206618 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:13:04.206589 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc85b7d1f_4d4b_4fdd_9427_d1c25942aebe.slice/crio-e701a40187dfdc2d22d82b5818f2e562a463fa17e54fbbe1efdb939fbbce031a WatchSource:0}: Error finding container e701a40187dfdc2d22d82b5818f2e562a463fa17e54fbbe1efdb939fbbce031a: Status 404 returned error can't find the container with id e701a40187dfdc2d22d82b5818f2e562a463fa17e54fbbe1efdb939fbbce031a Jun 15 03:13:04.946493 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:04.946452 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vz9g7" event={"ID":"c85b7d1f-4d4b-4fdd-9427-d1c25942aebe","Type":"ContainerStarted","Data":"e701a40187dfdc2d22d82b5818f2e562a463fa17e54fbbe1efdb939fbbce031a"} Jun 15 03:13:05.951661 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:05.951634 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vz9g7" event={"ID":"c85b7d1f-4d4b-4fdd-9427-d1c25942aebe","Type":"ContainerStarted","Data":"21b860cdb2453edadb2ec24bf7e8b087cff9a5008d270e040a54358fe366ea53"} Jun 15 03:13:06.958054 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:06.958016 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vz9g7" event={"ID":"c85b7d1f-4d4b-4fdd-9427-d1c25942aebe","Type":"ContainerStarted","Data":"6fe5413bad5fa7b531afb74d32846b651876a36511322a465e769dcfeea489c7"} Jun 15 03:13:06.974482 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:06.974433 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vz9g7" podStartSLOduration=253.399403101 podStartE2EDuration="4m14.974416856s" totalImagesPullingTime="1.575013755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:13:06.973417302 +0000 UTC m=+255.284556293" watchObservedRunningTime="2026-06-15 03:13:06.974416856 +0000 UTC m=+255.285555885" Jun 15 03:13:20.005137 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:20.005104 2562 generic.go:356] "Generic (PLEG): container finished" podID="06bfd370-abe8-4b3a-a82c-41e7d0ade9c7" containerID="cf94dd2b2f8707c9e4eefcc5ef96db8c54b381523f9aebb96f1c64757dabe606" exitCode=0 Jun 15 03:13:20.005552 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:20.005178 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" event={"ID":"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7","Type":"ContainerDied","Data":"cf94dd2b2f8707c9e4eefcc5ef96db8c54b381523f9aebb96f1c64757dabe606"} Jun 15 03:13:20.005552 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:20.005519 2562 scope.go:117] "RemoveContainer" containerID="cf94dd2b2f8707c9e4eefcc5ef96db8c54b381523f9aebb96f1c64757dabe606" Jun 15 03:13:21.011188 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:21.011156 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5bbd86d6bd-dswhg" event={"ID":"06bfd370-abe8-4b3a-a82c-41e7d0ade9c7","Type":"ContainerStarted","Data":"44baf2d9ad7bcadcb5aa8836488fd2a8d29bd10c94aeb0c0b8e29a41dd1e602f"} Jun 15 03:13:22.854302 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:22.854268 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4b504416-ace4-481d-97c4-abe48c82896d/init-config-reloader/0.log" Jun 15 03:13:23.053981 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:23.053955 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4b504416-ace4-481d-97c4-abe48c82896d/alertmanager/0.log" Jun 15 03:13:23.254213 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:23.254184 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4b504416-ace4-481d-97c4-abe48c82896d/config-reloader/0.log" Jun 15 03:13:23.454355 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:23.454326 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4b504416-ace4-481d-97c4-abe48c82896d/kube-rbac-proxy-web/0.log" Jun 15 03:13:23.653572 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:23.653535 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4b504416-ace4-481d-97c4-abe48c82896d/kube-rbac-proxy/0.log" Jun 15 03:13:23.854040 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:23.854010 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4b504416-ace4-481d-97c4-abe48c82896d/kube-rbac-proxy-metric/0.log" Jun 15 03:13:24.053606 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:24.053515 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4b504416-ace4-481d-97c4-abe48c82896d/prom-label-proxy/0.log" Jun 15 03:13:25.253460 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:25.253425 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-589576dfc7-pzl7w_455c4a83-df24-49bb-b1a2-f427ed4a795f/monitoring-plugin/0.log" Jun 15 03:13:26.652967 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:26.652938 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hmqkw_423a0d92-fe2b-4a84-b581-1bec5edad495/init-textfile/0.log" Jun 15 03:13:26.855389 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:26.855361 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hmqkw_423a0d92-fe2b-4a84-b581-1bec5edad495/node-exporter/0.log" Jun 15 03:13:27.053090 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:27.053012 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hmqkw_423a0d92-fe2b-4a84-b581-1bec5edad495/kube-rbac-proxy/0.log" Jun 15 03:13:27.253487 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:27.253459 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6b8d965dd5-tnhbx_63801ac3-c391-4e2f-93ca-8f0a495d672d/kube-rbac-proxy-main/0.log" Jun 15 03:13:27.453953 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:27.453925 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6b8d965dd5-tnhbx_63801ac3-c391-4e2f-93ca-8f0a495d672d/kube-rbac-proxy-self/0.log" Jun 15 03:13:27.654765 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:27.654735 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6b8d965dd5-tnhbx_63801ac3-c391-4e2f-93ca-8f0a495d672d/openshift-state-metrics/0.log" Jun 15 03:13:27.854845 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:27.854803 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_790ce4b0-43e7-48a9-a24b-954ade22f05b/init-config-reloader/0.log" Jun 15 03:13:28.056270 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:28.056220 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_790ce4b0-43e7-48a9-a24b-954ade22f05b/prometheus/0.log" Jun 15 03:13:28.255040 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:28.254960 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_790ce4b0-43e7-48a9-a24b-954ade22f05b/config-reloader/0.log" Jun 15 03:13:28.453970 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:28.453944 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_790ce4b0-43e7-48a9-a24b-954ade22f05b/thanos-sidecar/0.log" Jun 15 03:13:28.654235 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:28.654190 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_790ce4b0-43e7-48a9-a24b-954ade22f05b/kube-rbac-proxy-web/0.log" Jun 15 03:13:28.853862 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:28.853818 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_790ce4b0-43e7-48a9-a24b-954ade22f05b/kube-rbac-proxy/0.log" Jun 15 03:13:29.054066 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:29.053983 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_790ce4b0-43e7-48a9-a24b-954ade22f05b/kube-rbac-proxy-thanos/0.log" Jun 15 03:13:33.056868 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:33.056820 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-694c955b9-xqnxf_3d282f55-36e3-45f9-9a11-505ef16fc6c6/download-server/0.log" Jun 15 03:13:33.574734 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:13:33.574698 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-nb5km" podUID="167a4bfe-f33d-4a1e-878c-8faebc6f61e7" Jun 15 03:13:34.059156 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:34.059127 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nb5km" Jun 15 03:13:34.453786 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:34.453756 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lj57j_ff9255c7-dc08-4be9-9087-b749284c6358/dns-node-resolver/0.log" Jun 15 03:13:35.853931 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:35.853902 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xhccz_7fe3e8d4-b5da-4511-a7f8-f28596c5f336/node-ca/0.log" Jun 15 03:13:36.452848 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:36.452803 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p9h4n_f61b9993-27b1-4a1d-b7bc-632802832528/serve-healthcheck-canary/0.log" Jun 15 03:13:36.547424 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:36.547382 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:13:36.549672 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:36.549652 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/167a4bfe-f33d-4a1e-878c-8faebc6f61e7-metrics-tls\") pod \"dns-default-nb5km\" (UID: \"167a4bfe-f33d-4a1e-878c-8faebc6f61e7\") " pod="openshift-dns/dns-default-nb5km" Jun 15 03:13:36.762538 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:36.762451 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b8jdx\"" Jun 15 03:13:36.770141 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:36.770117 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nb5km" Jun 15 03:13:36.942170 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:36.942098 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nb5km"] Jun 15 03:13:36.945127 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:13:36.945090 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod167a4bfe_f33d_4a1e_878c_8faebc6f61e7.slice/crio-c9dffce7d79e1a26a494b52dad089dd774608b7b5c9de87edee2889d56683846 WatchSource:0}: Error finding container c9dffce7d79e1a26a494b52dad089dd774608b7b5c9de87edee2889d56683846: Status 404 returned error can't find the container with id c9dffce7d79e1a26a494b52dad089dd774608b7b5c9de87edee2889d56683846 Jun 15 03:13:37.070876 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:37.070773 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nb5km" event={"ID":"167a4bfe-f33d-4a1e-878c-8faebc6f61e7","Type":"ContainerStarted","Data":"c9dffce7d79e1a26a494b52dad089dd774608b7b5c9de87edee2889d56683846"} Jun 15 03:13:40.090267 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:40.090233 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nb5km" event={"ID":"167a4bfe-f33d-4a1e-878c-8faebc6f61e7","Type":"ContainerStarted","Data":"f90607c9f1623d76262b763dd7f5b48866f88a7e16cc972c1c8f7a64986bb6ce"} Jun 15 03:13:40.090697 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:40.090304 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nb5km" event={"ID":"167a4bfe-f33d-4a1e-878c-8faebc6f61e7","Type":"ContainerStarted","Data":"babdf6537150ee2c54ec397304a6792f947df9e1136121d698cf6ae86b2f52b6"} Jun 15 03:13:40.090697 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:40.090470 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nb5km" Jun 15 03:13:50.098860 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:50.098810 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nb5km" Jun 15 03:13:50.116343 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:50.116281 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nb5km" podStartSLOduration=262.131980136 podStartE2EDuration="4m24.116266288s" totalImagesPullingTime="1.984286152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:09:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:13:40.110602805 +0000 UTC m=+288.421741796" watchObservedRunningTime="2026-06-15 03:13:50.116266288 +0000 UTC m=+298.427405277" Jun 15 03:13:51.996902 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:51.996876 2562 kubelet.go:1648] "Image garbage collection succeeded" Jun 15 03:13:52.659992 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:52.659957 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:13:52.678080 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:52.678056 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:13:53.155501 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:13:53.155473 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:06.437798 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:06.437761 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 15 03:14:06.438239 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:06.438057 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="alertmanager" containerID="cri-o://55502f1b3c2d8ac5da6eb43ead041c70a745ef1b89da32c2e9a1f1d8cd539af8" gracePeriod=120 Jun 15 03:14:06.438239 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:06.438147 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="kube-rbac-proxy-metric" containerID="cri-o://a3dbef18c6c0d37ddefeff2c2dc07af87f921b33c773940b8910ed322c7bcef2" gracePeriod=120 Jun 15 03:14:06.438239 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:06.438158 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="prom-label-proxy" containerID="cri-o://22a13ac11431b668e455f1483ad013325d2a1b69c74b515a75bea7e92b83f68a" gracePeriod=120 Jun 15 03:14:06.438239 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:06.438175 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="kube-rbac-proxy-web" containerID="cri-o://dcd6c540d6e259233c2be7dd63ef6f52c246a5953b89977b1e942366d41db1a7" gracePeriod=120 Jun 15 03:14:06.438441 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:06.438211 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="config-reloader" containerID="cri-o://9a5926518e11f03e1f674754c7d5b569c03a5c1f77e478db0b4ecf4f94a869e7" gracePeriod=120 Jun 15 03:14:06.438441 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:06.438227 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="kube-rbac-proxy" containerID="cri-o://b9e43e8d510f233bd0a7f4895a4abcbe2a4f0b07aa896911a6d1ccee6bb16472" gracePeriod=120 Jun 15 03:14:07.197419 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.197388 2562 generic.go:356] "Generic (PLEG): container finished" podID="4b504416-ace4-481d-97c4-abe48c82896d" containerID="22a13ac11431b668e455f1483ad013325d2a1b69c74b515a75bea7e92b83f68a" exitCode=0 Jun 15 03:14:07.197419 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.197411 2562 generic.go:356] "Generic (PLEG): container finished" podID="4b504416-ace4-481d-97c4-abe48c82896d" containerID="b9e43e8d510f233bd0a7f4895a4abcbe2a4f0b07aa896911a6d1ccee6bb16472" exitCode=0 Jun 15 03:14:07.197419 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.197417 2562 generic.go:356] "Generic (PLEG): container finished" podID="4b504416-ace4-481d-97c4-abe48c82896d" containerID="9a5926518e11f03e1f674754c7d5b569c03a5c1f77e478db0b4ecf4f94a869e7" exitCode=0 Jun 15 03:14:07.197419 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.197423 2562 generic.go:356] "Generic (PLEG): container finished" podID="4b504416-ace4-481d-97c4-abe48c82896d" containerID="55502f1b3c2d8ac5da6eb43ead041c70a745ef1b89da32c2e9a1f1d8cd539af8" exitCode=0 Jun 15 03:14:07.197675 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.197464 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b504416-ace4-481d-97c4-abe48c82896d","Type":"ContainerDied","Data":"22a13ac11431b668e455f1483ad013325d2a1b69c74b515a75bea7e92b83f68a"} Jun 15 03:14:07.197675 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.197530 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b504416-ace4-481d-97c4-abe48c82896d","Type":"ContainerDied","Data":"b9e43e8d510f233bd0a7f4895a4abcbe2a4f0b07aa896911a6d1ccee6bb16472"} Jun 15 03:14:07.197675 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.197556 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b504416-ace4-481d-97c4-abe48c82896d","Type":"ContainerDied","Data":"9a5926518e11f03e1f674754c7d5b569c03a5c1f77e478db0b4ecf4f94a869e7"} Jun 15 03:14:07.197675 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.197579 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b504416-ace4-481d-97c4-abe48c82896d","Type":"ContainerDied","Data":"55502f1b3c2d8ac5da6eb43ead041c70a745ef1b89da32c2e9a1f1d8cd539af8"} Jun 15 03:14:07.687431 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.687407 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:07.824077 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.823986 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-cluster-tls-config\") pod \"4b504416-ace4-481d-97c4-abe48c82896d\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " Jun 15 03:14:07.824077 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.824026 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b504416-ace4-481d-97c4-abe48c82896d-alertmanager-trusted-ca-bundle\") pod \"4b504416-ace4-481d-97c4-abe48c82896d\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " Jun 15 03:14:07.824077 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.824066 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4b504416-ace4-481d-97c4-abe48c82896d-tls-assets\") pod \"4b504416-ace4-481d-97c4-abe48c82896d\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " Jun 15 03:14:07.824364 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.824086 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-main-tls\") pod \"4b504416-ace4-481d-97c4-abe48c82896d\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " Jun 15 03:14:07.824364 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.824128 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-web-config\") pod \"4b504416-ace4-481d-97c4-abe48c82896d\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " Jun 15 03:14:07.824364 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.824157 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b504416-ace4-481d-97c4-abe48c82896d-metrics-client-ca\") pod \"4b504416-ace4-481d-97c4-abe48c82896d\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " Jun 15 03:14:07.824364 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.824231 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4snq\" (UniqueName: \"kubernetes.io/projected/4b504416-ace4-481d-97c4-abe48c82896d-kube-api-access-b4snq\") pod \"4b504416-ace4-481d-97c4-abe48c82896d\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " Jun 15 03:14:07.824364 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.824272 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-kube-rbac-proxy\") pod \"4b504416-ace4-481d-97c4-abe48c82896d\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " Jun 15 03:14:07.824364 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.824316 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-config-volume\") pod \"4b504416-ace4-481d-97c4-abe48c82896d\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " Jun 15 03:14:07.824364 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.824347 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4b504416-ace4-481d-97c4-abe48c82896d-alertmanager-main-db\") pod \"4b504416-ace4-481d-97c4-abe48c82896d\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " Jun 15 03:14:07.824769 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.824377 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4b504416-ace4-481d-97c4-abe48c82896d-config-out\") pod \"4b504416-ace4-481d-97c4-abe48c82896d\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " Jun 15 03:14:07.824769 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.824401 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"4b504416-ace4-481d-97c4-abe48c82896d\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " Jun 15 03:14:07.824769 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.824429 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-kube-rbac-proxy-web\") pod \"4b504416-ace4-481d-97c4-abe48c82896d\" (UID: \"4b504416-ace4-481d-97c4-abe48c82896d\") " Jun 15 03:14:07.824769 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.824574 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b504416-ace4-481d-97c4-abe48c82896d-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "4b504416-ace4-481d-97c4-abe48c82896d" (UID: "4b504416-ace4-481d-97c4-abe48c82896d"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:14:07.824769 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.824746 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b504416-ace4-481d-97c4-abe48c82896d-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:07.825055 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.824914 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b504416-ace4-481d-97c4-abe48c82896d-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "4b504416-ace4-481d-97c4-abe48c82896d" (UID: "4b504416-ace4-481d-97c4-abe48c82896d"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:14:07.825306 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.825234 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b504416-ace4-481d-97c4-abe48c82896d-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "4b504416-ace4-481d-97c4-abe48c82896d" (UID: "4b504416-ace4-481d-97c4-abe48c82896d"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:14:07.827289 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.827205 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b504416-ace4-481d-97c4-abe48c82896d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4b504416-ace4-481d-97c4-abe48c82896d" (UID: "4b504416-ace4-481d-97c4-abe48c82896d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:14:07.827475 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.827448 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b504416-ace4-481d-97c4-abe48c82896d-kube-api-access-b4snq" (OuterVolumeSpecName: "kube-api-access-b4snq") pod "4b504416-ace4-481d-97c4-abe48c82896d" (UID: "4b504416-ace4-481d-97c4-abe48c82896d"). InnerVolumeSpecName "kube-api-access-b4snq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:14:07.827617 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.827567 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "4b504416-ace4-481d-97c4-abe48c82896d" (UID: "4b504416-ace4-481d-97c4-abe48c82896d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:14:07.827815 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.827792 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "4b504416-ace4-481d-97c4-abe48c82896d" (UID: "4b504416-ace4-481d-97c4-abe48c82896d"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:14:07.828389 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.828337 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "4b504416-ace4-481d-97c4-abe48c82896d" (UID: "4b504416-ace4-481d-97c4-abe48c82896d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:14:07.828519 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.828416 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-config-volume" (OuterVolumeSpecName: "config-volume") pod "4b504416-ace4-481d-97c4-abe48c82896d" (UID: "4b504416-ace4-481d-97c4-abe48c82896d"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:14:07.828519 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.828471 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "4b504416-ace4-481d-97c4-abe48c82896d" (UID: "4b504416-ace4-481d-97c4-abe48c82896d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:14:07.829093 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.829072 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b504416-ace4-481d-97c4-abe48c82896d-config-out" (OuterVolumeSpecName: "config-out") pod "4b504416-ace4-481d-97c4-abe48c82896d" (UID: "4b504416-ace4-481d-97c4-abe48c82896d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:14:07.832027 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.831941 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "4b504416-ace4-481d-97c4-abe48c82896d" (UID: "4b504416-ace4-481d-97c4-abe48c82896d"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:14:07.838593 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.838568 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-web-config" (OuterVolumeSpecName: "web-config") pod "4b504416-ace4-481d-97c4-abe48c82896d" (UID: "4b504416-ace4-481d-97c4-abe48c82896d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:14:07.925818 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.925780 2562 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-cluster-tls-config\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:07.925818 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.925812 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4b504416-ace4-481d-97c4-abe48c82896d-tls-assets\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:07.925818 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.925823 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-main-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:07.926064 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.925852 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-web-config\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:07.926064 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.925861 2562 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b504416-ace4-481d-97c4-abe48c82896d-metrics-client-ca\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:07.926064 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.925869 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4snq\" (UniqueName: \"kubernetes.io/projected/4b504416-ace4-481d-97c4-abe48c82896d-kube-api-access-b4snq\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:07.926064 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.925880 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:07.926064 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.925889 2562 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-config-volume\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:07.926064 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.925897 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4b504416-ace4-481d-97c4-abe48c82896d-alertmanager-main-db\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:07.926064 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.925905 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4b504416-ace4-481d-97c4-abe48c82896d-config-out\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:07.926064 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.925913 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:07.926064 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:07.925922 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b504416-ace4-481d-97c4-abe48c82896d-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:08.204506 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.204467 2562 generic.go:356] "Generic (PLEG): container finished" podID="4b504416-ace4-481d-97c4-abe48c82896d" containerID="a3dbef18c6c0d37ddefeff2c2dc07af87f921b33c773940b8910ed322c7bcef2" exitCode=0 Jun 15 03:14:08.204506 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.204499 2562 generic.go:356] "Generic (PLEG): container finished" podID="4b504416-ace4-481d-97c4-abe48c82896d" containerID="dcd6c540d6e259233c2be7dd63ef6f52c246a5953b89977b1e942366d41db1a7" exitCode=0 Jun 15 03:14:08.204705 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.204549 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b504416-ace4-481d-97c4-abe48c82896d","Type":"ContainerDied","Data":"a3dbef18c6c0d37ddefeff2c2dc07af87f921b33c773940b8910ed322c7bcef2"} Jun 15 03:14:08.204705 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.204621 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b504416-ace4-481d-97c4-abe48c82896d","Type":"ContainerDied","Data":"dcd6c540d6e259233c2be7dd63ef6f52c246a5953b89977b1e942366d41db1a7"} Jun 15 03:14:08.204705 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.204558 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 15 03:14:08.204705 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.204649 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b504416-ace4-481d-97c4-abe48c82896d","Type":"ContainerDied","Data":"c6479fc4b075daafce820f0043c3f12754bff1e1e1ecd632a514310bc46d8f9a"} Jun 15 03:14:08.204705 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.204680 2562 scope.go:117] "RemoveContainer" containerID="22a13ac11431b668e455f1483ad013325d2a1b69c74b515a75bea7e92b83f68a" Jun 15 03:14:08.212705 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.212686 2562 scope.go:117] "RemoveContainer" containerID="a3dbef18c6c0d37ddefeff2c2dc07af87f921b33c773940b8910ed322c7bcef2" Jun 15 03:14:08.219349 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.219321 2562 scope.go:117] "RemoveContainer" containerID="b9e43e8d510f233bd0a7f4895a4abcbe2a4f0b07aa896911a6d1ccee6bb16472" Jun 15 03:14:08.225083 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.225059 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 15 03:14:08.227130 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.227111 2562 scope.go:117] "RemoveContainer" containerID="dcd6c540d6e259233c2be7dd63ef6f52c246a5953b89977b1e942366d41db1a7" Jun 15 03:14:08.229298 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.229278 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 15 03:14:08.234535 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.234496 2562 scope.go:117] "RemoveContainer" containerID="9a5926518e11f03e1f674754c7d5b569c03a5c1f77e478db0b4ecf4f94a869e7" Jun 15 03:14:08.241298 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.241282 2562 scope.go:117] "RemoveContainer" containerID="55502f1b3c2d8ac5da6eb43ead041c70a745ef1b89da32c2e9a1f1d8cd539af8" Jun 15 03:14:08.248154 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.248131 2562 scope.go:117] "RemoveContainer" containerID="f684b23f9d5cc7d8c4d2e64c95a0410bd50673e53c9c3c800ee2ef51ff17b5d1" Jun 15 03:14:08.254685 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.254661 2562 scope.go:117] "RemoveContainer" containerID="22a13ac11431b668e455f1483ad013325d2a1b69c74b515a75bea7e92b83f68a" Jun 15 03:14:08.254975 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:14:08.254945 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22a13ac11431b668e455f1483ad013325d2a1b69c74b515a75bea7e92b83f68a\": container with ID starting with 22a13ac11431b668e455f1483ad013325d2a1b69c74b515a75bea7e92b83f68a not found: ID does not exist" containerID="22a13ac11431b668e455f1483ad013325d2a1b69c74b515a75bea7e92b83f68a" Jun 15 03:14:08.255065 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.254988 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22a13ac11431b668e455f1483ad013325d2a1b69c74b515a75bea7e92b83f68a"} err="failed to get container status \"22a13ac11431b668e455f1483ad013325d2a1b69c74b515a75bea7e92b83f68a\": rpc error: code = NotFound desc = could not find container \"22a13ac11431b668e455f1483ad013325d2a1b69c74b515a75bea7e92b83f68a\": container with ID starting with 22a13ac11431b668e455f1483ad013325d2a1b69c74b515a75bea7e92b83f68a not found: ID does not exist" Jun 15 03:14:08.255065 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.255016 2562 scope.go:117] "RemoveContainer" containerID="a3dbef18c6c0d37ddefeff2c2dc07af87f921b33c773940b8910ed322c7bcef2" Jun 15 03:14:08.255496 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:14:08.255310 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3dbef18c6c0d37ddefeff2c2dc07af87f921b33c773940b8910ed322c7bcef2\": container with ID starting with a3dbef18c6c0d37ddefeff2c2dc07af87f921b33c773940b8910ed322c7bcef2 not found: ID does not exist" containerID="a3dbef18c6c0d37ddefeff2c2dc07af87f921b33c773940b8910ed322c7bcef2" Jun 15 03:14:08.255496 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.255349 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3dbef18c6c0d37ddefeff2c2dc07af87f921b33c773940b8910ed322c7bcef2"} err="failed to get container status \"a3dbef18c6c0d37ddefeff2c2dc07af87f921b33c773940b8910ed322c7bcef2\": rpc error: code = NotFound desc = could not find container \"a3dbef18c6c0d37ddefeff2c2dc07af87f921b33c773940b8910ed322c7bcef2\": container with ID starting with a3dbef18c6c0d37ddefeff2c2dc07af87f921b33c773940b8910ed322c7bcef2 not found: ID does not exist" Jun 15 03:14:08.255669 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.255563 2562 scope.go:117] "RemoveContainer" containerID="b9e43e8d510f233bd0a7f4895a4abcbe2a4f0b07aa896911a6d1ccee6bb16472" Jun 15 03:14:08.255855 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:14:08.255822 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9e43e8d510f233bd0a7f4895a4abcbe2a4f0b07aa896911a6d1ccee6bb16472\": container with ID starting with b9e43e8d510f233bd0a7f4895a4abcbe2a4f0b07aa896911a6d1ccee6bb16472 not found: ID does not exist" containerID="b9e43e8d510f233bd0a7f4895a4abcbe2a4f0b07aa896911a6d1ccee6bb16472" Jun 15 03:14:08.255912 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.255863 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9e43e8d510f233bd0a7f4895a4abcbe2a4f0b07aa896911a6d1ccee6bb16472"} err="failed to get container status \"b9e43e8d510f233bd0a7f4895a4abcbe2a4f0b07aa896911a6d1ccee6bb16472\": rpc error: code = NotFound desc = could not find container \"b9e43e8d510f233bd0a7f4895a4abcbe2a4f0b07aa896911a6d1ccee6bb16472\": container with ID starting with b9e43e8d510f233bd0a7f4895a4abcbe2a4f0b07aa896911a6d1ccee6bb16472 not found: ID does not exist" Jun 15 03:14:08.255912 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.255878 2562 scope.go:117] "RemoveContainer" containerID="dcd6c540d6e259233c2be7dd63ef6f52c246a5953b89977b1e942366d41db1a7" Jun 15 03:14:08.256089 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:14:08.256075 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcd6c540d6e259233c2be7dd63ef6f52c246a5953b89977b1e942366d41db1a7\": container with ID starting with dcd6c540d6e259233c2be7dd63ef6f52c246a5953b89977b1e942366d41db1a7 not found: ID does not exist" containerID="dcd6c540d6e259233c2be7dd63ef6f52c246a5953b89977b1e942366d41db1a7" Jun 15 03:14:08.256124 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.256092 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcd6c540d6e259233c2be7dd63ef6f52c246a5953b89977b1e942366d41db1a7"} err="failed to get container status \"dcd6c540d6e259233c2be7dd63ef6f52c246a5953b89977b1e942366d41db1a7\": rpc error: code = NotFound desc = could not find container \"dcd6c540d6e259233c2be7dd63ef6f52c246a5953b89977b1e942366d41db1a7\": container with ID starting with dcd6c540d6e259233c2be7dd63ef6f52c246a5953b89977b1e942366d41db1a7 not found: ID does not exist" Jun 15 03:14:08.256124 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.256106 2562 scope.go:117] "RemoveContainer" containerID="9a5926518e11f03e1f674754c7d5b569c03a5c1f77e478db0b4ecf4f94a869e7" Jun 15 03:14:08.256298 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:14:08.256280 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a5926518e11f03e1f674754c7d5b569c03a5c1f77e478db0b4ecf4f94a869e7\": container with ID starting with 9a5926518e11f03e1f674754c7d5b569c03a5c1f77e478db0b4ecf4f94a869e7 not found: ID does not exist" containerID="9a5926518e11f03e1f674754c7d5b569c03a5c1f77e478db0b4ecf4f94a869e7" Jun 15 03:14:08.256332 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.256303 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a5926518e11f03e1f674754c7d5b569c03a5c1f77e478db0b4ecf4f94a869e7"} err="failed to get container status \"9a5926518e11f03e1f674754c7d5b569c03a5c1f77e478db0b4ecf4f94a869e7\": rpc error: code = NotFound desc = could not find container \"9a5926518e11f03e1f674754c7d5b569c03a5c1f77e478db0b4ecf4f94a869e7\": container with ID starting with 9a5926518e11f03e1f674754c7d5b569c03a5c1f77e478db0b4ecf4f94a869e7 not found: ID does not exist" Jun 15 03:14:08.256332 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.256315 2562 scope.go:117] "RemoveContainer" containerID="55502f1b3c2d8ac5da6eb43ead041c70a745ef1b89da32c2e9a1f1d8cd539af8" Jun 15 03:14:08.256549 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:14:08.256530 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55502f1b3c2d8ac5da6eb43ead041c70a745ef1b89da32c2e9a1f1d8cd539af8\": container with ID starting with 55502f1b3c2d8ac5da6eb43ead041c70a745ef1b89da32c2e9a1f1d8cd539af8 not found: ID does not exist" containerID="55502f1b3c2d8ac5da6eb43ead041c70a745ef1b89da32c2e9a1f1d8cd539af8" Jun 15 03:14:08.256593 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.256555 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55502f1b3c2d8ac5da6eb43ead041c70a745ef1b89da32c2e9a1f1d8cd539af8"} err="failed to get container status \"55502f1b3c2d8ac5da6eb43ead041c70a745ef1b89da32c2e9a1f1d8cd539af8\": rpc error: code = NotFound desc = could not find container \"55502f1b3c2d8ac5da6eb43ead041c70a745ef1b89da32c2e9a1f1d8cd539af8\": container with ID starting with 55502f1b3c2d8ac5da6eb43ead041c70a745ef1b89da32c2e9a1f1d8cd539af8 not found: ID does not exist" Jun 15 03:14:08.256593 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.256571 2562 scope.go:117] "RemoveContainer" containerID="f684b23f9d5cc7d8c4d2e64c95a0410bd50673e53c9c3c800ee2ef51ff17b5d1" Jun 15 03:14:08.256808 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:14:08.256790 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f684b23f9d5cc7d8c4d2e64c95a0410bd50673e53c9c3c800ee2ef51ff17b5d1\": container with ID starting with f684b23f9d5cc7d8c4d2e64c95a0410bd50673e53c9c3c800ee2ef51ff17b5d1 not found: ID does not exist" containerID="f684b23f9d5cc7d8c4d2e64c95a0410bd50673e53c9c3c800ee2ef51ff17b5d1" Jun 15 03:14:08.256869 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.256812 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f684b23f9d5cc7d8c4d2e64c95a0410bd50673e53c9c3c800ee2ef51ff17b5d1"} err="failed to get container status \"f684b23f9d5cc7d8c4d2e64c95a0410bd50673e53c9c3c800ee2ef51ff17b5d1\": rpc error: code = NotFound desc = could not find container \"f684b23f9d5cc7d8c4d2e64c95a0410bd50673e53c9c3c800ee2ef51ff17b5d1\": container with ID starting with f684b23f9d5cc7d8c4d2e64c95a0410bd50673e53c9c3c800ee2ef51ff17b5d1 not found: ID does not exist" Jun 15 03:14:08.256869 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.256848 2562 scope.go:117] "RemoveContainer" containerID="22a13ac11431b668e455f1483ad013325d2a1b69c74b515a75bea7e92b83f68a" Jun 15 03:14:08.257044 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.257029 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22a13ac11431b668e455f1483ad013325d2a1b69c74b515a75bea7e92b83f68a"} err="failed to get container status \"22a13ac11431b668e455f1483ad013325d2a1b69c74b515a75bea7e92b83f68a\": rpc error: code = NotFound desc = could not find container \"22a13ac11431b668e455f1483ad013325d2a1b69c74b515a75bea7e92b83f68a\": container with ID starting with 22a13ac11431b668e455f1483ad013325d2a1b69c74b515a75bea7e92b83f68a not found: ID does not exist" Jun 15 03:14:08.257086 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.257043 2562 scope.go:117] "RemoveContainer" containerID="a3dbef18c6c0d37ddefeff2c2dc07af87f921b33c773940b8910ed322c7bcef2" Jun 15 03:14:08.257264 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.257249 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3dbef18c6c0d37ddefeff2c2dc07af87f921b33c773940b8910ed322c7bcef2"} err="failed to get container status \"a3dbef18c6c0d37ddefeff2c2dc07af87f921b33c773940b8910ed322c7bcef2\": rpc error: code = NotFound desc = could not find container \"a3dbef18c6c0d37ddefeff2c2dc07af87f921b33c773940b8910ed322c7bcef2\": container with ID starting with a3dbef18c6c0d37ddefeff2c2dc07af87f921b33c773940b8910ed322c7bcef2 not found: ID does not exist" Jun 15 03:14:08.257305 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.257265 2562 scope.go:117] "RemoveContainer" containerID="b9e43e8d510f233bd0a7f4895a4abcbe2a4f0b07aa896911a6d1ccee6bb16472" Jun 15 03:14:08.257458 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.257443 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9e43e8d510f233bd0a7f4895a4abcbe2a4f0b07aa896911a6d1ccee6bb16472"} err="failed to get container status \"b9e43e8d510f233bd0a7f4895a4abcbe2a4f0b07aa896911a6d1ccee6bb16472\": rpc error: code = NotFound desc = could not find container \"b9e43e8d510f233bd0a7f4895a4abcbe2a4f0b07aa896911a6d1ccee6bb16472\": container with ID starting with b9e43e8d510f233bd0a7f4895a4abcbe2a4f0b07aa896911a6d1ccee6bb16472 not found: ID does not exist" Jun 15 03:14:08.257499 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.257458 2562 scope.go:117] "RemoveContainer" containerID="dcd6c540d6e259233c2be7dd63ef6f52c246a5953b89977b1e942366d41db1a7" Jun 15 03:14:08.257675 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.257653 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcd6c540d6e259233c2be7dd63ef6f52c246a5953b89977b1e942366d41db1a7"} err="failed to get container status \"dcd6c540d6e259233c2be7dd63ef6f52c246a5953b89977b1e942366d41db1a7\": rpc error: code = NotFound desc = could not find container \"dcd6c540d6e259233c2be7dd63ef6f52c246a5953b89977b1e942366d41db1a7\": container with ID starting with dcd6c540d6e259233c2be7dd63ef6f52c246a5953b89977b1e942366d41db1a7 not found: ID does not exist" Jun 15 03:14:08.257729 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.257675 2562 scope.go:117] "RemoveContainer" containerID="9a5926518e11f03e1f674754c7d5b569c03a5c1f77e478db0b4ecf4f94a869e7" Jun 15 03:14:08.257904 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.257887 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a5926518e11f03e1f674754c7d5b569c03a5c1f77e478db0b4ecf4f94a869e7"} err="failed to get container status \"9a5926518e11f03e1f674754c7d5b569c03a5c1f77e478db0b4ecf4f94a869e7\": rpc error: code = NotFound desc = could not find container \"9a5926518e11f03e1f674754c7d5b569c03a5c1f77e478db0b4ecf4f94a869e7\": container with ID starting with 9a5926518e11f03e1f674754c7d5b569c03a5c1f77e478db0b4ecf4f94a869e7 not found: ID does not exist" Jun 15 03:14:08.257944 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.257906 2562 scope.go:117] "RemoveContainer" containerID="55502f1b3c2d8ac5da6eb43ead041c70a745ef1b89da32c2e9a1f1d8cd539af8" Jun 15 03:14:08.258124 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.258108 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55502f1b3c2d8ac5da6eb43ead041c70a745ef1b89da32c2e9a1f1d8cd539af8"} err="failed to get container status \"55502f1b3c2d8ac5da6eb43ead041c70a745ef1b89da32c2e9a1f1d8cd539af8\": rpc error: code = NotFound desc = could not find container \"55502f1b3c2d8ac5da6eb43ead041c70a745ef1b89da32c2e9a1f1d8cd539af8\": container with ID starting with 55502f1b3c2d8ac5da6eb43ead041c70a745ef1b89da32c2e9a1f1d8cd539af8 not found: ID does not exist" Jun 15 03:14:08.258166 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.258125 2562 scope.go:117] "RemoveContainer" containerID="f684b23f9d5cc7d8c4d2e64c95a0410bd50673e53c9c3c800ee2ef51ff17b5d1" Jun 15 03:14:08.258318 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:08.258299 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f684b23f9d5cc7d8c4d2e64c95a0410bd50673e53c9c3c800ee2ef51ff17b5d1"} err="failed to get container status \"f684b23f9d5cc7d8c4d2e64c95a0410bd50673e53c9c3c800ee2ef51ff17b5d1\": rpc error: code = NotFound desc = could not find container \"f684b23f9d5cc7d8c4d2e64c95a0410bd50673e53c9c3c800ee2ef51ff17b5d1\": container with ID starting with f684b23f9d5cc7d8c4d2e64c95a0410bd50673e53c9c3c800ee2ef51ff17b5d1 not found: ID does not exist" Jun 15 03:14:10.044260 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:10.044228 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b504416-ace4-481d-97c4-abe48c82896d" path="/var/lib/kubelet/pods/4b504416-ace4-481d-97c4-abe48c82896d/volumes" Jun 15 03:14:10.750700 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:10.750664 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 15 03:14:10.751200 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:10.751040 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="prometheus" containerID="cri-o://4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960" gracePeriod=600 Jun 15 03:14:10.751200 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:10.751058 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="kube-rbac-proxy" containerID="cri-o://cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a" gracePeriod=600 Jun 15 03:14:10.751200 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:10.751096 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="kube-rbac-proxy-thanos" containerID="cri-o://e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e" gracePeriod=600 Jun 15 03:14:10.751200 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:10.751059 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="thanos-sidecar" containerID="cri-o://acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae" gracePeriod=600 Jun 15 03:14:10.751200 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:10.751069 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="config-reloader" containerID="cri-o://5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e" gracePeriod=600 Jun 15 03:14:10.751200 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:10.751096 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="kube-rbac-proxy-web" containerID="cri-o://1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8" gracePeriod=600 Jun 15 03:14:11.012986 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.012962 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.151494 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.151452 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"790ce4b0-43e7-48a9-a24b-954ade22f05b\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " Jun 15 03:14:11.151494 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.151493 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-rulefiles-0\") pod \"790ce4b0-43e7-48a9-a24b-954ade22f05b\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " Jun 15 03:14:11.152053 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.151515 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-configmap-metrics-client-ca\") pod \"790ce4b0-43e7-48a9-a24b-954ade22f05b\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " Jun 15 03:14:11.152053 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.151530 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-thanos-prometheus-http-client-file\") pod \"790ce4b0-43e7-48a9-a24b-954ade22f05b\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " Jun 15 03:14:11.152053 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.151563 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"790ce4b0-43e7-48a9-a24b-954ade22f05b\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " Jun 15 03:14:11.152053 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.151586 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kflj6\" (UniqueName: \"kubernetes.io/projected/790ce4b0-43e7-48a9-a24b-954ade22f05b-kube-api-access-kflj6\") pod \"790ce4b0-43e7-48a9-a24b-954ade22f05b\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " Jun 15 03:14:11.152053 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.151621 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-config\") pod \"790ce4b0-43e7-48a9-a24b-954ade22f05b\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " Jun 15 03:14:11.152053 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.151655 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-trusted-ca-bundle\") pod \"790ce4b0-43e7-48a9-a24b-954ade22f05b\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " Jun 15 03:14:11.152053 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.151683 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-prometheus-k8s-tls\") pod \"790ce4b0-43e7-48a9-a24b-954ade22f05b\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " Jun 15 03:14:11.152053 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.151709 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-metrics-client-certs\") pod \"790ce4b0-43e7-48a9-a24b-954ade22f05b\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " Jun 15 03:14:11.152053 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.151740 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-rulefiles-2\") pod \"790ce4b0-43e7-48a9-a24b-954ade22f05b\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " Jun 15 03:14:11.152053 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.151771 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/790ce4b0-43e7-48a9-a24b-954ade22f05b-config-out\") pod \"790ce4b0-43e7-48a9-a24b-954ade22f05b\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " Jun 15 03:14:11.152053 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.151804 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/790ce4b0-43e7-48a9-a24b-954ade22f05b-tls-assets\") pod \"790ce4b0-43e7-48a9-a24b-954ade22f05b\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " Jun 15 03:14:11.152053 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.151860 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-kube-rbac-proxy\") pod \"790ce4b0-43e7-48a9-a24b-954ade22f05b\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " Jun 15 03:14:11.152053 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.151892 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-web-config\") pod \"790ce4b0-43e7-48a9-a24b-954ade22f05b\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " Jun 15 03:14:11.152053 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.151923 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-configmap-kubelet-serving-ca-bundle\") pod \"790ce4b0-43e7-48a9-a24b-954ade22f05b\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " Jun 15 03:14:11.152053 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.151934 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "790ce4b0-43e7-48a9-a24b-954ade22f05b" (UID: "790ce4b0-43e7-48a9-a24b-954ade22f05b"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:14:11.152053 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.151953 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-rulefiles-1\") pod \"790ce4b0-43e7-48a9-a24b-954ade22f05b\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " Jun 15 03:14:11.152053 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.152018 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-db\") pod \"790ce4b0-43e7-48a9-a24b-954ade22f05b\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " Jun 15 03:14:11.152053 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.152051 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-grpc-tls\") pod \"790ce4b0-43e7-48a9-a24b-954ade22f05b\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " Jun 15 03:14:11.152931 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.152078 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-configmap-serving-certs-ca-bundle\") pod \"790ce4b0-43e7-48a9-a24b-954ade22f05b\" (UID: \"790ce4b0-43e7-48a9-a24b-954ade22f05b\") " Jun 15 03:14:11.152931 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.152200 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "790ce4b0-43e7-48a9-a24b-954ade22f05b" (UID: "790ce4b0-43e7-48a9-a24b-954ade22f05b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:14:11.152931 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.152330 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-configmap-metrics-client-ca\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:11.152931 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.152350 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:11.152931 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.152704 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "790ce4b0-43e7-48a9-a24b-954ade22f05b" (UID: "790ce4b0-43e7-48a9-a24b-954ade22f05b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:14:11.152931 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.152899 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "790ce4b0-43e7-48a9-a24b-954ade22f05b" (UID: "790ce4b0-43e7-48a9-a24b-954ade22f05b"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:14:11.154112 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.153595 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "790ce4b0-43e7-48a9-a24b-954ade22f05b" (UID: "790ce4b0-43e7-48a9-a24b-954ade22f05b"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:14:11.154112 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.153859 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "790ce4b0-43e7-48a9-a24b-954ade22f05b" (UID: "790ce4b0-43e7-48a9-a24b-954ade22f05b"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:14:11.154562 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.154511 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "790ce4b0-43e7-48a9-a24b-954ade22f05b" (UID: "790ce4b0-43e7-48a9-a24b-954ade22f05b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:14:11.154660 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.154640 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "790ce4b0-43e7-48a9-a24b-954ade22f05b" (UID: "790ce4b0-43e7-48a9-a24b-954ade22f05b"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:14:11.154746 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.154724 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "790ce4b0-43e7-48a9-a24b-954ade22f05b" (UID: "790ce4b0-43e7-48a9-a24b-954ade22f05b"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:14:11.154813 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.154778 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "790ce4b0-43e7-48a9-a24b-954ade22f05b" (UID: "790ce4b0-43e7-48a9-a24b-954ade22f05b"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:14:11.154905 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.154867 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "790ce4b0-43e7-48a9-a24b-954ade22f05b" (UID: "790ce4b0-43e7-48a9-a24b-954ade22f05b"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:14:11.155035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.155010 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "790ce4b0-43e7-48a9-a24b-954ade22f05b" (UID: "790ce4b0-43e7-48a9-a24b-954ade22f05b"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:14:11.156320 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.156292 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/790ce4b0-43e7-48a9-a24b-954ade22f05b-kube-api-access-kflj6" (OuterVolumeSpecName: "kube-api-access-kflj6") pod "790ce4b0-43e7-48a9-a24b-954ade22f05b" (UID: "790ce4b0-43e7-48a9-a24b-954ade22f05b"). InnerVolumeSpecName "kube-api-access-kflj6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:14:11.156320 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.156296 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/790ce4b0-43e7-48a9-a24b-954ade22f05b-config-out" (OuterVolumeSpecName: "config-out") pod "790ce4b0-43e7-48a9-a24b-954ade22f05b" (UID: "790ce4b0-43e7-48a9-a24b-954ade22f05b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:14:11.156581 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.156554 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "790ce4b0-43e7-48a9-a24b-954ade22f05b" (UID: "790ce4b0-43e7-48a9-a24b-954ade22f05b"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:14:11.156700 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.156641 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-config" (OuterVolumeSpecName: "config") pod "790ce4b0-43e7-48a9-a24b-954ade22f05b" (UID: "790ce4b0-43e7-48a9-a24b-954ade22f05b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:14:11.156755 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.156734 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "790ce4b0-43e7-48a9-a24b-954ade22f05b" (UID: "790ce4b0-43e7-48a9-a24b-954ade22f05b"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:14:11.157028 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.157007 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "790ce4b0-43e7-48a9-a24b-954ade22f05b" (UID: "790ce4b0-43e7-48a9-a24b-954ade22f05b"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:14:11.157551 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.157530 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/790ce4b0-43e7-48a9-a24b-954ade22f05b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "790ce4b0-43e7-48a9-a24b-954ade22f05b" (UID: "790ce4b0-43e7-48a9-a24b-954ade22f05b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:14:11.165417 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.165394 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-web-config" (OuterVolumeSpecName: "web-config") pod "790ce4b0-43e7-48a9-a24b-954ade22f05b" (UID: "790ce4b0-43e7-48a9-a24b-954ade22f05b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 15 03:14:11.220548 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.220518 2562 generic.go:356] "Generic (PLEG): container finished" podID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerID="e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e" exitCode=0 Jun 15 03:14:11.220548 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.220542 2562 generic.go:356] "Generic (PLEG): container finished" podID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerID="cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a" exitCode=0 Jun 15 03:14:11.220548 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.220549 2562 generic.go:356] "Generic (PLEG): container finished" podID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerID="1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8" exitCode=0 Jun 15 03:14:11.220548 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.220555 2562 generic.go:356] "Generic (PLEG): container finished" podID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerID="acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae" exitCode=0 Jun 15 03:14:11.220890 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.220560 2562 generic.go:356] "Generic (PLEG): container finished" podID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerID="5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e" exitCode=0 Jun 15 03:14:11.220890 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.220565 2562 generic.go:356] "Generic (PLEG): container finished" podID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerID="4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960" exitCode=0 Jun 15 03:14:11.220890 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.220587 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.220890 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.220596 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"790ce4b0-43e7-48a9-a24b-954ade22f05b","Type":"ContainerDied","Data":"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e"} Jun 15 03:14:11.220890 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.220680 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"790ce4b0-43e7-48a9-a24b-954ade22f05b","Type":"ContainerDied","Data":"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a"} Jun 15 03:14:11.220890 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.220722 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"790ce4b0-43e7-48a9-a24b-954ade22f05b","Type":"ContainerDied","Data":"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8"} Jun 15 03:14:11.220890 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.220742 2562 scope.go:117] "RemoveContainer" containerID="e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e" Jun 15 03:14:11.220890 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.220766 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"790ce4b0-43e7-48a9-a24b-954ade22f05b","Type":"ContainerDied","Data":"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae"} Jun 15 03:14:11.220890 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.220810 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"790ce4b0-43e7-48a9-a24b-954ade22f05b","Type":"ContainerDied","Data":"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e"} Jun 15 03:14:11.221286 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.220980 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"790ce4b0-43e7-48a9-a24b-954ade22f05b","Type":"ContainerDied","Data":"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960"} Jun 15 03:14:11.221286 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.221027 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"790ce4b0-43e7-48a9-a24b-954ade22f05b","Type":"ContainerDied","Data":"27d37857ae3d3c397ddeb5916d9aac9a29a828528f72991fe3b69b9e55707e71"} Jun 15 03:14:11.228365 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.228200 2562 scope.go:117] "RemoveContainer" containerID="cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a" Jun 15 03:14:11.235043 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.235026 2562 scope.go:117] "RemoveContainer" containerID="1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8" Jun 15 03:14:11.241478 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.241460 2562 scope.go:117] "RemoveContainer" containerID="acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae" Jun 15 03:14:11.244317 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.244297 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 15 03:14:11.248510 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.248489 2562 scope.go:117] "RemoveContainer" containerID="5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e" Jun 15 03:14:11.248677 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.248661 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 15 03:14:11.252672 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.252653 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:11.252742 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.252674 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:11.252742 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.252684 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-db\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:11.252742 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.252694 2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-grpc-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:11.252742 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.252702 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:11.252742 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.252711 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:11.252742 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.252720 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:11.252742 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.252728 2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:11.252742 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.252737 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:11.253038 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.252745 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kflj6\" (UniqueName: \"kubernetes.io/projected/790ce4b0-43e7-48a9-a24b-954ade22f05b-kube-api-access-kflj6\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:11.253038 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.252754 2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-config\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:11.253038 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.252762 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/790ce4b0-43e7-48a9-a24b-954ade22f05b-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:11.253038 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.252770 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:11.253038 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.252779 2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-metrics-client-certs\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:11.253038 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.252788 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/790ce4b0-43e7-48a9-a24b-954ade22f05b-config-out\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:11.253038 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.252796 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/790ce4b0-43e7-48a9-a24b-954ade22f05b-tls-assets\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:11.253038 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.252804 2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-secret-kube-rbac-proxy\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:11.253038 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.252852 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/790ce4b0-43e7-48a9-a24b-954ade22f05b-web-config\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:14:11.254859 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.254822 2562 scope.go:117] "RemoveContainer" containerID="4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960" Jun 15 03:14:11.261598 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.261582 2562 scope.go:117] "RemoveContainer" containerID="e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7" Jun 15 03:14:11.267862 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.267828 2562 scope.go:117] "RemoveContainer" containerID="e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e" Jun 15 03:14:11.268126 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:14:11.268107 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e\": container with ID starting with e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e not found: ID does not exist" containerID="e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e" Jun 15 03:14:11.268176 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.268135 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e"} err="failed to get container status \"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e\": rpc error: code = NotFound desc = could not find container \"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e\": container with ID starting with e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e not found: ID does not exist" Jun 15 03:14:11.268176 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.268155 2562 scope.go:117] "RemoveContainer" containerID="cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a" Jun 15 03:14:11.268393 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:14:11.268376 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a\": container with ID starting with cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a not found: ID does not exist" containerID="cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a" Jun 15 03:14:11.268437 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.268399 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a"} err="failed to get container status \"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a\": rpc error: code = NotFound desc = could not find container \"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a\": container with ID starting with cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a not found: ID does not exist" Jun 15 03:14:11.268437 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.268416 2562 scope.go:117] "RemoveContainer" containerID="1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8" Jun 15 03:14:11.268619 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:14:11.268603 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8\": container with ID starting with 1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8 not found: ID does not exist" containerID="1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8" Jun 15 03:14:11.268680 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.268626 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8"} err="failed to get container status \"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8\": rpc error: code = NotFound desc = could not find container \"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8\": container with ID starting with 1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8 not found: ID does not exist" Jun 15 03:14:11.268680 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.268649 2562 scope.go:117] "RemoveContainer" containerID="acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae" Jun 15 03:14:11.268901 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:14:11.268884 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae\": container with ID starting with acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae not found: ID does not exist" containerID="acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae" Jun 15 03:14:11.268947 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.268906 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae"} err="failed to get container status \"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae\": rpc error: code = NotFound desc = could not find container \"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae\": container with ID starting with acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae not found: ID does not exist" Jun 15 03:14:11.268947 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.268921 2562 scope.go:117] "RemoveContainer" containerID="5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e" Jun 15 03:14:11.269143 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:14:11.269121 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e\": container with ID starting with 5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e not found: ID does not exist" containerID="5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e" Jun 15 03:14:11.269185 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.269148 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e"} err="failed to get container status \"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e\": rpc error: code = NotFound desc = could not find container \"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e\": container with ID starting with 5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e not found: ID does not exist" Jun 15 03:14:11.269185 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.269164 2562 scope.go:117] "RemoveContainer" containerID="4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960" Jun 15 03:14:11.269401 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:14:11.269385 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960\": container with ID starting with 4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960 not found: ID does not exist" containerID="4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960" Jun 15 03:14:11.269448 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.269406 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960"} err="failed to get container status \"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960\": rpc error: code = NotFound desc = could not find container \"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960\": container with ID starting with 4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960 not found: ID does not exist" Jun 15 03:14:11.269448 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.269419 2562 scope.go:117] "RemoveContainer" containerID="e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7" Jun 15 03:14:11.269612 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:14:11.269597 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7\": container with ID starting with e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7 not found: ID does not exist" containerID="e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7" Jun 15 03:14:11.269654 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.269614 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7"} err="failed to get container status \"e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7\": rpc error: code = NotFound desc = could not find container \"e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7\": container with ID starting with e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7 not found: ID does not exist" Jun 15 03:14:11.269654 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.269626 2562 scope.go:117] "RemoveContainer" containerID="e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e" Jun 15 03:14:11.269820 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.269800 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e"} err="failed to get container status \"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e\": rpc error: code = NotFound desc = could not find container \"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e\": container with ID starting with e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e not found: ID does not exist" Jun 15 03:14:11.269917 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.269822 2562 scope.go:117] "RemoveContainer" containerID="cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a" Jun 15 03:14:11.270084 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.270067 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a"} err="failed to get container status \"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a\": rpc error: code = NotFound desc = could not find container \"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a\": container with ID starting with cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a not found: ID does not exist" Jun 15 03:14:11.270132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.270084 2562 scope.go:117] "RemoveContainer" containerID="1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8" Jun 15 03:14:11.270294 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.270270 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8"} err="failed to get container status \"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8\": rpc error: code = NotFound desc = could not find container \"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8\": container with ID starting with 1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8 not found: ID does not exist" Jun 15 03:14:11.270294 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.270289 2562 scope.go:117] "RemoveContainer" containerID="acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae" Jun 15 03:14:11.270550 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.270529 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae"} err="failed to get container status \"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae\": rpc error: code = NotFound desc = could not find container \"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae\": container with ID starting with acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae not found: ID does not exist" Jun 15 03:14:11.270621 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.270552 2562 scope.go:117] "RemoveContainer" containerID="5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e" Jun 15 03:14:11.270760 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.270746 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e"} err="failed to get container status \"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e\": rpc error: code = NotFound desc = could not find container \"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e\": container with ID starting with 5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e not found: ID does not exist" Jun 15 03:14:11.270805 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.270760 2562 scope.go:117] "RemoveContainer" containerID="4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960" Jun 15 03:14:11.270977 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.270959 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960"} err="failed to get container status \"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960\": rpc error: code = NotFound desc = could not find container \"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960\": container with ID starting with 4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960 not found: ID does not exist" Jun 15 03:14:11.271030 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.270978 2562 scope.go:117] "RemoveContainer" containerID="e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7" Jun 15 03:14:11.271197 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.271178 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7"} err="failed to get container status \"e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7\": rpc error: code = NotFound desc = could not find container \"e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7\": container with ID starting with e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7 not found: ID does not exist" Jun 15 03:14:11.271258 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.271198 2562 scope.go:117] "RemoveContainer" containerID="e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e" Jun 15 03:14:11.271412 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.271393 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e"} err="failed to get container status \"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e\": rpc error: code = NotFound desc = could not find container \"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e\": container with ID starting with e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e not found: ID does not exist" Jun 15 03:14:11.271457 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.271413 2562 scope.go:117] "RemoveContainer" containerID="cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a" Jun 15 03:14:11.271628 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.271610 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a"} err="failed to get container status \"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a\": rpc error: code = NotFound desc = could not find container \"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a\": container with ID starting with cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a not found: ID does not exist" Jun 15 03:14:11.271673 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.271628 2562 scope.go:117] "RemoveContainer" containerID="1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8" Jun 15 03:14:11.271822 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.271805 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8"} err="failed to get container status \"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8\": rpc error: code = NotFound desc = could not find container \"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8\": container with ID starting with 1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8 not found: ID does not exist" Jun 15 03:14:11.271917 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.271824 2562 scope.go:117] "RemoveContainer" containerID="acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae" Jun 15 03:14:11.272035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.272019 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae"} err="failed to get container status \"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae\": rpc error: code = NotFound desc = could not find container \"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae\": container with ID starting with acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae not found: ID does not exist" Jun 15 03:14:11.272078 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.272034 2562 scope.go:117] "RemoveContainer" containerID="5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e" Jun 15 03:14:11.272240 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.272223 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e"} err="failed to get container status \"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e\": rpc error: code = NotFound desc = could not find container \"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e\": container with ID starting with 5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e not found: ID does not exist" Jun 15 03:14:11.272287 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.272240 2562 scope.go:117] "RemoveContainer" containerID="4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960" Jun 15 03:14:11.272455 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.272439 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960"} err="failed to get container status \"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960\": rpc error: code = NotFound desc = could not find container \"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960\": container with ID starting with 4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960 not found: ID does not exist" Jun 15 03:14:11.272455 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.272454 2562 scope.go:117] "RemoveContainer" containerID="e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7" Jun 15 03:14:11.272637 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.272620 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7"} err="failed to get container status \"e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7\": rpc error: code = NotFound desc = could not find container \"e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7\": container with ID starting with e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7 not found: ID does not exist" Jun 15 03:14:11.272679 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.272638 2562 scope.go:117] "RemoveContainer" containerID="e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e" Jun 15 03:14:11.272849 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.272814 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e"} err="failed to get container status \"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e\": rpc error: code = NotFound desc = could not find container \"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e\": container with ID starting with e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e not found: ID does not exist" Jun 15 03:14:11.272900 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.272850 2562 scope.go:117] "RemoveContainer" containerID="cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a" Jun 15 03:14:11.273040 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.273024 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a"} err="failed to get container status \"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a\": rpc error: code = NotFound desc = could not find container \"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a\": container with ID starting with cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a not found: ID does not exist" Jun 15 03:14:11.273088 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.273043 2562 scope.go:117] "RemoveContainer" containerID="1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8" Jun 15 03:14:11.273246 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.273229 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8"} err="failed to get container status \"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8\": rpc error: code = NotFound desc = could not find container \"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8\": container with ID starting with 1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8 not found: ID does not exist" Jun 15 03:14:11.273292 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.273247 2562 scope.go:117] "RemoveContainer" containerID="acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae" Jun 15 03:14:11.273463 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.273445 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae"} err="failed to get container status \"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae\": rpc error: code = NotFound desc = could not find container \"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae\": container with ID starting with acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae not found: ID does not exist" Jun 15 03:14:11.273508 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.273466 2562 scope.go:117] "RemoveContainer" containerID="5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e" Jun 15 03:14:11.273675 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.273657 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e"} err="failed to get container status \"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e\": rpc error: code = NotFound desc = could not find container \"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e\": container with ID starting with 5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e not found: ID does not exist" Jun 15 03:14:11.273745 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.273678 2562 scope.go:117] "RemoveContainer" containerID="4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960" Jun 15 03:14:11.273912 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.273893 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960"} err="failed to get container status \"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960\": rpc error: code = NotFound desc = could not find container \"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960\": container with ID starting with 4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960 not found: ID does not exist" Jun 15 03:14:11.273955 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.273912 2562 scope.go:117] "RemoveContainer" containerID="e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7" Jun 15 03:14:11.274140 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.274123 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7"} err="failed to get container status \"e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7\": rpc error: code = NotFound desc = could not find container \"e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7\": container with ID starting with e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7 not found: ID does not exist" Jun 15 03:14:11.274199 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.274141 2562 scope.go:117] "RemoveContainer" containerID="e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e" Jun 15 03:14:11.274348 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.274333 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e"} err="failed to get container status \"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e\": rpc error: code = NotFound desc = could not find container \"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e\": container with ID starting with e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e not found: ID does not exist" Jun 15 03:14:11.274397 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.274350 2562 scope.go:117] "RemoveContainer" containerID="cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a" Jun 15 03:14:11.274504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.274489 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a"} err="failed to get container status \"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a\": rpc error: code = NotFound desc = could not find container \"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a\": container with ID starting with cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a not found: ID does not exist" Jun 15 03:14:11.274504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.274503 2562 scope.go:117] "RemoveContainer" containerID="1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8" Jun 15 03:14:11.274709 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.274685 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8"} err="failed to get container status \"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8\": rpc error: code = NotFound desc = could not find container \"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8\": container with ID starting with 1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8 not found: ID does not exist" Jun 15 03:14:11.274709 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.274709 2562 scope.go:117] "RemoveContainer" containerID="acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae" Jun 15 03:14:11.275091 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.275065 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae"} err="failed to get container status \"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae\": rpc error: code = NotFound desc = could not find container \"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae\": container with ID starting with acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae not found: ID does not exist" Jun 15 03:14:11.275188 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.275096 2562 scope.go:117] "RemoveContainer" containerID="5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e" Jun 15 03:14:11.275484 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.275398 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e"} err="failed to get container status \"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e\": rpc error: code = NotFound desc = could not find container \"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e\": container with ID starting with 5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e not found: ID does not exist" Jun 15 03:14:11.275484 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.275421 2562 scope.go:117] "RemoveContainer" containerID="4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960" Jun 15 03:14:11.275662 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.275641 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960"} err="failed to get container status \"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960\": rpc error: code = NotFound desc = could not find container \"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960\": container with ID starting with 4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960 not found: ID does not exist" Jun 15 03:14:11.275662 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.275661 2562 scope.go:117] "RemoveContainer" containerID="e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7" Jun 15 03:14:11.275922 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.275897 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7"} err="failed to get container status \"e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7\": rpc error: code = NotFound desc = could not find container \"e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7\": container with ID starting with e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7 not found: ID does not exist" Jun 15 03:14:11.275922 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.275919 2562 scope.go:117] "RemoveContainer" containerID="e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e" Jun 15 03:14:11.276174 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.276152 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e"} err="failed to get container status \"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e\": rpc error: code = NotFound desc = could not find container \"e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e\": container with ID starting with e22ee9cf9cd38e9084f5e29f0665b7fefb7f621f5ef67d8b7d047012d9878a5e not found: ID does not exist" Jun 15 03:14:11.276221 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.276177 2562 scope.go:117] "RemoveContainer" containerID="cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a" Jun 15 03:14:11.276430 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.276407 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a"} err="failed to get container status \"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a\": rpc error: code = NotFound desc = could not find container \"cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a\": container with ID starting with cee31e9b3799a8423eef781a7b9ecfa107f09a991b6ec9acb100158fbc9cf32a not found: ID does not exist" Jun 15 03:14:11.276511 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.276431 2562 scope.go:117] "RemoveContainer" containerID="1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8" Jun 15 03:14:11.276679 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.276658 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8"} err="failed to get container status \"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8\": rpc error: code = NotFound desc = could not find container \"1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8\": container with ID starting with 1b039934fd0f4779d08e761d987796f0328c0e199e04035080e45730900f13d8 not found: ID does not exist" Jun 15 03:14:11.276731 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.276682 2562 scope.go:117] "RemoveContainer" containerID="acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae" Jun 15 03:14:11.276782 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.276724 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 15 03:14:11.276955 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.276935 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae"} err="failed to get container status \"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae\": rpc error: code = NotFound desc = could not find container \"acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae\": container with ID starting with acfe09aacc1ad2af7f2fa9e525f774dc627feb3da51354cc5d60bea59695ccae not found: ID does not exist" Jun 15 03:14:11.277012 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.276956 2562 scope.go:117] "RemoveContainer" containerID="5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e" Jun 15 03:14:11.277012 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.276990 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="kube-rbac-proxy-metric" Jun 15 03:14:11.277012 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277001 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="kube-rbac-proxy-metric" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277017 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="kube-rbac-proxy" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277025 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="kube-rbac-proxy" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277034 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="kube-rbac-proxy-web" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277040 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="kube-rbac-proxy-web" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277047 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="kube-rbac-proxy" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277052 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="kube-rbac-proxy" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277058 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="kube-rbac-proxy-thanos" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277063 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="kube-rbac-proxy-thanos" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277071 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="thanos-sidecar" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277076 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="thanos-sidecar" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277081 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="prom-label-proxy" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277089 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="prom-label-proxy" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277098 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="prometheus" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277103 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="prometheus" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277109 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="config-reloader" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277115 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="config-reloader" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277121 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="kube-rbac-proxy-web" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277126 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="kube-rbac-proxy-web" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277133 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="init-config-reloader" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277139 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="init-config-reloader" Jun 15 03:14:11.277132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277143 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="init-config-reloader" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277140 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e"} err="failed to get container status \"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e\": rpc error: code = NotFound desc = could not find container \"5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e\": container with ID starting with 5fa81a9d649fb9da244cefa3f60a7e8f810c59c9e1e20910bef60cb1e4c8822e not found: ID does not exist" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277160 2562 scope.go:117] "RemoveContainer" containerID="4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277148 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="init-config-reloader" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277205 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="alertmanager" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277212 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="alertmanager" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277221 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="config-reloader" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277227 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="config-reloader" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277308 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="thanos-sidecar" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277319 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="prometheus" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277327 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="kube-rbac-proxy" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277338 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="config-reloader" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277345 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="kube-rbac-proxy" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277352 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="prom-label-proxy" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277360 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="kube-rbac-proxy-web" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277366 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="kube-rbac-proxy-metric" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277371 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="config-reloader" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277378 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b504416-ace4-481d-97c4-abe48c82896d" containerName="alertmanager" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277383 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="kube-rbac-proxy-web" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277387 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" containerName="kube-rbac-proxy-thanos" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277414 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960"} err="failed to get container status \"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960\": rpc error: code = NotFound desc = could not find container \"4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960\": container with ID starting with 4b0468c68b023c1f1d6204e7e1d49d69f83fc3ec460a4b6dbf96e0bcaa9bd960 not found: ID does not exist" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277430 2562 scope.go:117] "RemoveContainer" containerID="e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7" Jun 15 03:14:11.277773 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.277637 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7"} err="failed to get container status \"e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7\": rpc error: code = NotFound desc = could not find container \"e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7\": container with ID starting with e2b9c526882d72e6ba204eec9899f5ac59f1821863028685a3cbe837f62839a7 not found: ID does not exist" Jun 15 03:14:11.282309 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.282295 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.284537 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.284518 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 15 03:14:11.284647 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.284546 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 15 03:14:11.284647 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.284546 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 15 03:14:11.284647 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.284622 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 15 03:14:11.284800 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.284546 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 15 03:14:11.284800 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.284722 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 15 03:14:11.285095 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.285077 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 15 03:14:11.285163 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.285098 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 15 03:14:11.285163 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.285081 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7ooqann5mhn16\"" Jun 15 03:14:11.285267 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.285194 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-t2cln\"" Jun 15 03:14:11.285402 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.285386 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 15 03:14:11.285442 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.285416 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 15 03:14:11.285744 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.285730 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 15 03:14:11.285917 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.285901 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 15 03:14:11.286016 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.285990 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 15 03:14:11.288255 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.288237 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 15 03:14:11.291324 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.291305 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 15 03:14:11.298137 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.298117 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 15 03:14:11.455352 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.455305 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.455352 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.455352 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.455573 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.455374 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.455573 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.455426 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-web-config\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.455573 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.455465 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.455573 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.455487 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.455573 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.455508 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-config\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.455573 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.455529 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.455573 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.455569 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.455775 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.455592 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.455775 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.455615 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.455775 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.455637 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.455775 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.455656 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ba6152df-a718-417e-b907-540d1ba9079a-config-out\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.455775 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.455686 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.455775 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.455707 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92xr7\" (UniqueName: \"kubernetes.io/projected/ba6152df-a718-417e-b907-540d1ba9079a-kube-api-access-92xr7\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.455775 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.455726 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.455775 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.455752 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ba6152df-a718-417e-b907-540d1ba9079a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.456035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.455790 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.456035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.455807 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.456035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.455827 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ba6152df-a718-417e-b907-540d1ba9079a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.556701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.556613 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-config\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.556701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.556667 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.556701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.556692 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.556701 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.556710 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.556977 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.556726 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.556977 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.556741 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.556977 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.556761 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ba6152df-a718-417e-b907-540d1ba9079a-config-out\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.556977 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.556783 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.556977 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.556797 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92xr7\" (UniqueName: \"kubernetes.io/projected/ba6152df-a718-417e-b907-540d1ba9079a-kube-api-access-92xr7\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.558133 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.557581 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.558133 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.557641 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.558133 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.557687 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ba6152df-a718-417e-b907-540d1ba9079a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.558133 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.557773 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.558133 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.557727 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.558133 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.557974 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.558133 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.558001 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ba6152df-a718-417e-b907-540d1ba9079a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.558133 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.558078 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.558133 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.558110 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.558133 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.558137 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.558695 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.558162 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-web-config\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.558695 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.558206 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.558695 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.558207 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.558695 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.558232 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.558951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.558928 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.559463 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.559432 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.560059 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.559708 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ba6152df-a718-417e-b907-540d1ba9079a-config-out\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.560059 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.559785 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-config\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.560059 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.559948 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.560059 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.560014 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.561205 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.560870 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.561205 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.560881 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.561205 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.561143 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ba6152df-a718-417e-b907-540d1ba9079a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.561453 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.561428 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.561515 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.561478 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba6152df-a718-417e-b907-540d1ba9079a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.562149 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.562123 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.562236 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.562126 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ba6152df-a718-417e-b907-540d1ba9079a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.562351 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.562332 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.562430 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.562410 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-web-config\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.562526 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.562510 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ba6152df-a718-417e-b907-540d1ba9079a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.563996 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.563981 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92xr7\" (UniqueName: \"kubernetes.io/projected/ba6152df-a718-417e-b907-540d1ba9079a-kube-api-access-92xr7\") pod \"prometheus-k8s-0\" (UID: \"ba6152df-a718-417e-b907-540d1ba9079a\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.591952 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.591930 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:14:11.722418 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:11.722384 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 15 03:14:11.725358 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:14:11.725332 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba6152df_a718_417e_b907_540d1ba9079a.slice/crio-047c6db5b5078b53fa3a8cb431e5b5b094613b9ea526c56299245c31ee8495dc WatchSource:0}: Error finding container 047c6db5b5078b53fa3a8cb431e5b5b094613b9ea526c56299245c31ee8495dc: Status 404 returned error can't find the container with id 047c6db5b5078b53fa3a8cb431e5b5b094613b9ea526c56299245c31ee8495dc Jun 15 03:14:12.043779 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:12.043747 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="790ce4b0-43e7-48a9-a24b-954ade22f05b" path="/var/lib/kubelet/pods/790ce4b0-43e7-48a9-a24b-954ade22f05b/volumes" Jun 15 03:14:12.226489 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:12.226450 2562 generic.go:356] "Generic (PLEG): container finished" podID="ba6152df-a718-417e-b907-540d1ba9079a" containerID="09063a742cfed53d2248c378ead65b0828af8e3eda5d42aea47ef5bd88d519e1" exitCode=0 Jun 15 03:14:12.226915 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:12.226514 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ba6152df-a718-417e-b907-540d1ba9079a","Type":"ContainerDied","Data":"09063a742cfed53d2248c378ead65b0828af8e3eda5d42aea47ef5bd88d519e1"} Jun 15 03:14:12.226915 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:12.226579 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ba6152df-a718-417e-b907-540d1ba9079a","Type":"ContainerStarted","Data":"047c6db5b5078b53fa3a8cb431e5b5b094613b9ea526c56299245c31ee8495dc"} Jun 15 03:14:13.233522 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:13.233487 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ba6152df-a718-417e-b907-540d1ba9079a","Type":"ContainerStarted","Data":"c719310d2b23f473e03616998e8b208998a2f168d52e00e173699c16674870d3"} Jun 15 03:14:13.233908 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:13.233603 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ba6152df-a718-417e-b907-540d1ba9079a","Type":"ContainerStarted","Data":"99d7102cca42c5ead6406486a263f7b067410ac0446fea379094a83935b43653"} Jun 15 03:14:13.233908 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:13.233650 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ba6152df-a718-417e-b907-540d1ba9079a","Type":"ContainerStarted","Data":"8a70aed562259c4d91965d60fd3f9665236f3486d11a42386603abb6060c1b7a"} Jun 15 03:14:13.233908 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:13.233686 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ba6152df-a718-417e-b907-540d1ba9079a","Type":"ContainerStarted","Data":"c9b9d13a5889dbb9a29d5dd2008fe1918f9c306240be112344478e4483aca151"} Jun 15 03:14:13.233908 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:13.233720 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ba6152df-a718-417e-b907-540d1ba9079a","Type":"ContainerStarted","Data":"606099186c1d3ced5e6fd417bc3c7ddd4be2d2c500e0672e5032185b965c5d7a"} Jun 15 03:14:13.233908 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:13.233781 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ba6152df-a718-417e-b907-540d1ba9079a","Type":"ContainerStarted","Data":"47ddc52ad3d31986e65b2863067e72813e7e3deb79c216976105070aad2d6976"} Jun 15 03:14:13.260943 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:13.260882 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.241827453 podStartE2EDuration="2.260861098s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.033645ms" isStatefulPod=false podCreationTimestamp="2026-06-15 03:14:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:14:13.259254735 +0000 UTC m=+321.570393748" watchObservedRunningTime="2026-06-15 03:14:13.260861098 +0000 UTC m=+321.572000090" Jun 15 03:14:16.592677 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:14:16.592625 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:15:11.593162 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:11.593114 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:15:11.608287 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:11.608259 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:15:12.464776 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:12.464748 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 15 03:15:56.870619 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:56.870582 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd"] Jun 15 03:15:56.874046 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:56.874029 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" Jun 15 03:15:56.879185 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:56.879146 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 15 03:15:56.879326 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:56.879204 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 15 03:15:56.879326 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:56.879301 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 15 03:15:56.879326 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:56.879317 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-tzlp8\"" Jun 15 03:15:56.879495 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:56.879390 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 15 03:15:56.889173 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:56.889148 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd"] Jun 15 03:15:56.949255 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:56.949219 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df113ca2-256b-46fb-aec0-8693e938e085-webhook-cert\") pod \"opendatahub-operator-controller-manager-5799f64c8f-fk5zd\" (UID: \"df113ca2-256b-46fb-aec0-8693e938e085\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" Jun 15 03:15:56.949432 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:56.949285 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df113ca2-256b-46fb-aec0-8693e938e085-tmp\") pod \"opendatahub-operator-controller-manager-5799f64c8f-fk5zd\" (UID: \"df113ca2-256b-46fb-aec0-8693e938e085\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" Jun 15 03:15:56.949432 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:56.949329 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df113ca2-256b-46fb-aec0-8693e938e085-manifests\") pod \"opendatahub-operator-controller-manager-5799f64c8f-fk5zd\" (UID: \"df113ca2-256b-46fb-aec0-8693e938e085\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" Jun 15 03:15:56.949432 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:56.949345 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4c72\" (UniqueName: \"kubernetes.io/projected/df113ca2-256b-46fb-aec0-8693e938e085-kube-api-access-x4c72\") pod \"opendatahub-operator-controller-manager-5799f64c8f-fk5zd\" (UID: \"df113ca2-256b-46fb-aec0-8693e938e085\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" Jun 15 03:15:56.949533 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:56.949438 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df113ca2-256b-46fb-aec0-8693e938e085-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5799f64c8f-fk5zd\" (UID: \"df113ca2-256b-46fb-aec0-8693e938e085\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" Jun 15 03:15:57.050478 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:57.050436 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df113ca2-256b-46fb-aec0-8693e938e085-webhook-cert\") pod \"opendatahub-operator-controller-manager-5799f64c8f-fk5zd\" (UID: \"df113ca2-256b-46fb-aec0-8693e938e085\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" Jun 15 03:15:57.050478 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:57.050487 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df113ca2-256b-46fb-aec0-8693e938e085-tmp\") pod \"opendatahub-operator-controller-manager-5799f64c8f-fk5zd\" (UID: \"df113ca2-256b-46fb-aec0-8693e938e085\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" Jun 15 03:15:57.050687 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:57.050517 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df113ca2-256b-46fb-aec0-8693e938e085-manifests\") pod \"opendatahub-operator-controller-manager-5799f64c8f-fk5zd\" (UID: \"df113ca2-256b-46fb-aec0-8693e938e085\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" Jun 15 03:15:57.050687 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:57.050535 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4c72\" (UniqueName: \"kubernetes.io/projected/df113ca2-256b-46fb-aec0-8693e938e085-kube-api-access-x4c72\") pod \"opendatahub-operator-controller-manager-5799f64c8f-fk5zd\" (UID: \"df113ca2-256b-46fb-aec0-8693e938e085\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" Jun 15 03:15:57.050687 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:57.050562 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df113ca2-256b-46fb-aec0-8693e938e085-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5799f64c8f-fk5zd\" (UID: \"df113ca2-256b-46fb-aec0-8693e938e085\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" Jun 15 03:15:57.051043 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:57.051015 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df113ca2-256b-46fb-aec0-8693e938e085-manifests\") pod \"opendatahub-operator-controller-manager-5799f64c8f-fk5zd\" (UID: \"df113ca2-256b-46fb-aec0-8693e938e085\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" Jun 15 03:15:57.052782 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:57.052760 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df113ca2-256b-46fb-aec0-8693e938e085-tmp\") pod \"opendatahub-operator-controller-manager-5799f64c8f-fk5zd\" (UID: \"df113ca2-256b-46fb-aec0-8693e938e085\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" Jun 15 03:15:57.052932 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:57.052914 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df113ca2-256b-46fb-aec0-8693e938e085-webhook-cert\") pod \"opendatahub-operator-controller-manager-5799f64c8f-fk5zd\" (UID: \"df113ca2-256b-46fb-aec0-8693e938e085\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" Jun 15 03:15:57.053011 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:57.052996 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df113ca2-256b-46fb-aec0-8693e938e085-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5799f64c8f-fk5zd\" (UID: \"df113ca2-256b-46fb-aec0-8693e938e085\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" Jun 15 03:15:57.061892 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:57.061863 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4c72\" (UniqueName: \"kubernetes.io/projected/df113ca2-256b-46fb-aec0-8693e938e085-kube-api-access-x4c72\") pod \"opendatahub-operator-controller-manager-5799f64c8f-fk5zd\" (UID: \"df113ca2-256b-46fb-aec0-8693e938e085\") " pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" Jun 15 03:15:57.184332 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:57.184238 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" Jun 15 03:15:57.323690 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:57.323652 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd"] Jun 15 03:15:57.327983 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:15:57.327947 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf113ca2_256b_46fb_aec0_8693e938e085.slice/crio-6683518db64eb4f3576d97a1b9e685e40b6b8fabe4e8daec8e628f4f0f24ab65 WatchSource:0}: Error finding container 6683518db64eb4f3576d97a1b9e685e40b6b8fabe4e8daec8e628f4f0f24ab65: Status 404 returned error can't find the container with id 6683518db64eb4f3576d97a1b9e685e40b6b8fabe4e8daec8e628f4f0f24ab65 Jun 15 03:15:57.329752 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:57.329733 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 15 03:15:57.605359 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:15:57.605329 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" event={"ID":"df113ca2-256b-46fb-aec0-8693e938e085","Type":"ContainerStarted","Data":"6683518db64eb4f3576d97a1b9e685e40b6b8fabe4e8daec8e628f4f0f24ab65"} Jun 15 03:16:00.619031 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:00.618984 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" event={"ID":"df113ca2-256b-46fb-aec0-8693e938e085","Type":"ContainerStarted","Data":"1b483e6e7abdc62aafda74c2659c70cfa17104f1eb14ae208cad5bdb6907e35e"} Jun 15 03:16:01.626771 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:01.626731 2562 generic.go:356] "Generic (PLEG): container finished" podID="df113ca2-256b-46fb-aec0-8693e938e085" containerID="1b483e6e7abdc62aafda74c2659c70cfa17104f1eb14ae208cad5bdb6907e35e" exitCode=0 Jun 15 03:16:01.627229 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:01.626819 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" event={"ID":"df113ca2-256b-46fb-aec0-8693e938e085","Type":"ContainerDied","Data":"1b483e6e7abdc62aafda74c2659c70cfa17104f1eb14ae208cad5bdb6907e35e"} Jun 15 03:16:02.632456 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:02.632413 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" event={"ID":"df113ca2-256b-46fb-aec0-8693e938e085","Type":"ContainerStarted","Data":"90a2a5a30020773901b7ff0f2ecf81ec9936824f1bd438d49acd97e90671aac0"} Jun 15 03:16:02.632965 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:02.632565 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" Jun 15 03:16:02.651723 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:02.651675 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" podStartSLOduration=2.5463701800000003 podStartE2EDuration="6.651660269s" totalImagesPullingTime="3.170578987s" totalInitContainerRuntime="934.711102ms" isStatefulPod=false podCreationTimestamp="2026-06-15 03:15:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:16:02.649703008 +0000 UTC m=+430.960841999" watchObservedRunningTime="2026-06-15 03:16:02.651660269 +0000 UTC m=+430.962799258" Jun 15 03:16:13.638438 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:13.638361 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5799f64c8f-fk5zd" Jun 15 03:16:16.798285 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:16.798249 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9"] Jun 15 03:16:16.805126 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:16.805104 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9" Jun 15 03:16:16.807972 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:16.807949 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-fdfsz\"" Jun 15 03:16:16.808094 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:16.807972 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 15 03:16:16.808094 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:16.807997 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 15 03:16:16.808094 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:16.808005 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 15 03:16:16.808094 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:16.807951 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 15 03:16:16.808094 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:16.807951 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 15 03:16:16.812298 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:16.811942 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9"] Jun 15 03:16:16.912409 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:16.912380 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/790c85c2-7984-4567-b4a1-1083ad9b3b94-cert\") pod \"lws-controller-manager-6b4969d8b8-bk8p9\" (UID: \"790c85c2-7984-4567-b4a1-1083ad9b3b94\") " pod="openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9" Jun 15 03:16:16.912603 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:16.912416 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pthxk\" (UniqueName: \"kubernetes.io/projected/790c85c2-7984-4567-b4a1-1083ad9b3b94-kube-api-access-pthxk\") pod \"lws-controller-manager-6b4969d8b8-bk8p9\" (UID: \"790c85c2-7984-4567-b4a1-1083ad9b3b94\") " pod="openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9" Jun 15 03:16:16.912603 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:16.912445 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/790c85c2-7984-4567-b4a1-1083ad9b3b94-metrics-cert\") pod \"lws-controller-manager-6b4969d8b8-bk8p9\" (UID: \"790c85c2-7984-4567-b4a1-1083ad9b3b94\") " pod="openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9" Jun 15 03:16:16.912603 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:16.912528 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/790c85c2-7984-4567-b4a1-1083ad9b3b94-manager-config\") pod \"lws-controller-manager-6b4969d8b8-bk8p9\" (UID: \"790c85c2-7984-4567-b4a1-1083ad9b3b94\") " pod="openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9" Jun 15 03:16:17.013725 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:17.013690 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pthxk\" (UniqueName: \"kubernetes.io/projected/790c85c2-7984-4567-b4a1-1083ad9b3b94-kube-api-access-pthxk\") pod \"lws-controller-manager-6b4969d8b8-bk8p9\" (UID: \"790c85c2-7984-4567-b4a1-1083ad9b3b94\") " pod="openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9" Jun 15 03:16:17.013941 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:17.013745 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/790c85c2-7984-4567-b4a1-1083ad9b3b94-metrics-cert\") pod \"lws-controller-manager-6b4969d8b8-bk8p9\" (UID: \"790c85c2-7984-4567-b4a1-1083ad9b3b94\") " pod="openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9" Jun 15 03:16:17.013941 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:17.013781 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/790c85c2-7984-4567-b4a1-1083ad9b3b94-manager-config\") pod \"lws-controller-manager-6b4969d8b8-bk8p9\" (UID: \"790c85c2-7984-4567-b4a1-1083ad9b3b94\") " pod="openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9" Jun 15 03:16:17.013941 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:17.013871 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/790c85c2-7984-4567-b4a1-1083ad9b3b94-cert\") pod \"lws-controller-manager-6b4969d8b8-bk8p9\" (UID: \"790c85c2-7984-4567-b4a1-1083ad9b3b94\") " pod="openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9" Jun 15 03:16:17.014556 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:17.014527 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/790c85c2-7984-4567-b4a1-1083ad9b3b94-manager-config\") pod \"lws-controller-manager-6b4969d8b8-bk8p9\" (UID: \"790c85c2-7984-4567-b4a1-1083ad9b3b94\") " pod="openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9" Jun 15 03:16:17.016346 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:17.016329 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/790c85c2-7984-4567-b4a1-1083ad9b3b94-cert\") pod \"lws-controller-manager-6b4969d8b8-bk8p9\" (UID: \"790c85c2-7984-4567-b4a1-1083ad9b3b94\") " pod="openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9" Jun 15 03:16:17.016434 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:17.016366 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/790c85c2-7984-4567-b4a1-1083ad9b3b94-metrics-cert\") pod \"lws-controller-manager-6b4969d8b8-bk8p9\" (UID: \"790c85c2-7984-4567-b4a1-1083ad9b3b94\") " pod="openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9" Jun 15 03:16:17.022798 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:17.022765 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pthxk\" (UniqueName: \"kubernetes.io/projected/790c85c2-7984-4567-b4a1-1083ad9b3b94-kube-api-access-pthxk\") pod \"lws-controller-manager-6b4969d8b8-bk8p9\" (UID: \"790c85c2-7984-4567-b4a1-1083ad9b3b94\") " pod="openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9" Jun 15 03:16:17.114557 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:17.114522 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9" Jun 15 03:16:17.246990 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:17.246952 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9"] Jun 15 03:16:17.250513 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:16:17.250486 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod790c85c2_7984_4567_b4a1_1083ad9b3b94.slice/crio-5fa7c746ac3e72a07e40e1e76053c1622362f6977292abf8952375c3e741d013 WatchSource:0}: Error finding container 5fa7c746ac3e72a07e40e1e76053c1622362f6977292abf8952375c3e741d013: Status 404 returned error can't find the container with id 5fa7c746ac3e72a07e40e1e76053c1622362f6977292abf8952375c3e741d013 Jun 15 03:16:17.686775 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:17.686740 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9" event={"ID":"790c85c2-7984-4567-b4a1-1083ad9b3b94","Type":"ContainerStarted","Data":"5fa7c746ac3e72a07e40e1e76053c1622362f6977292abf8952375c3e741d013"} Jun 15 03:16:20.698935 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:20.698891 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9" event={"ID":"790c85c2-7984-4567-b4a1-1083ad9b3b94","Type":"ContainerStarted","Data":"fed18914897b9235361a1fe890a4ab7f9fc9a41739d4bcd9024f71f9a2f2f618"} Jun 15 03:16:20.699287 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:20.698983 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9" Jun 15 03:16:20.714848 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:20.714787 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9" podStartSLOduration=1.79843124 podStartE2EDuration="4.714765271s" totalImagesPullingTime="2.916334031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:16:20.713792032 +0000 UTC m=+449.024931016" watchObservedRunningTime="2026-06-15 03:16:20.714765271 +0000 UTC m=+449.025904260" Jun 15 03:16:31.704453 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:31.704422 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6b4969d8b8-bk8p9" Jun 15 03:16:38.072127 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:38.072093 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-694cb6497d-45zn5"] Jun 15 03:16:38.075574 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:38.075552 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-694cb6497d-45zn5" Jun 15 03:16:38.078438 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:38.078412 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-485sd\"" Jun 15 03:16:38.078544 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:38.078417 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 15 03:16:38.084627 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:38.084604 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-694cb6497d-45zn5"] Jun 15 03:16:38.201893 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:38.201821 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/277599cb-8940-479c-a249-5244193660d4-redirect-config\") pod \"dashboard-redirect-694cb6497d-45zn5\" (UID: \"277599cb-8940-479c-a249-5244193660d4\") " pod="opendatahub/dashboard-redirect-694cb6497d-45zn5" Jun 15 03:16:38.202086 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:38.201952 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s25hh\" (UniqueName: \"kubernetes.io/projected/277599cb-8940-479c-a249-5244193660d4-kube-api-access-s25hh\") pod \"dashboard-redirect-694cb6497d-45zn5\" (UID: \"277599cb-8940-479c-a249-5244193660d4\") " pod="opendatahub/dashboard-redirect-694cb6497d-45zn5" Jun 15 03:16:38.302857 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:38.302768 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s25hh\" (UniqueName: \"kubernetes.io/projected/277599cb-8940-479c-a249-5244193660d4-kube-api-access-s25hh\") pod \"dashboard-redirect-694cb6497d-45zn5\" (UID: \"277599cb-8940-479c-a249-5244193660d4\") " pod="opendatahub/dashboard-redirect-694cb6497d-45zn5" Jun 15 03:16:38.303122 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:38.302897 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/277599cb-8940-479c-a249-5244193660d4-redirect-config\") pod \"dashboard-redirect-694cb6497d-45zn5\" (UID: \"277599cb-8940-479c-a249-5244193660d4\") " pod="opendatahub/dashboard-redirect-694cb6497d-45zn5" Jun 15 03:16:38.303649 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:38.303623 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/277599cb-8940-479c-a249-5244193660d4-redirect-config\") pod \"dashboard-redirect-694cb6497d-45zn5\" (UID: \"277599cb-8940-479c-a249-5244193660d4\") " pod="opendatahub/dashboard-redirect-694cb6497d-45zn5" Jun 15 03:16:38.311251 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:38.311227 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s25hh\" (UniqueName: \"kubernetes.io/projected/277599cb-8940-479c-a249-5244193660d4-kube-api-access-s25hh\") pod \"dashboard-redirect-694cb6497d-45zn5\" (UID: \"277599cb-8940-479c-a249-5244193660d4\") " pod="opendatahub/dashboard-redirect-694cb6497d-45zn5" Jun 15 03:16:38.386654 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:38.386615 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-694cb6497d-45zn5" Jun 15 03:16:38.523798 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:38.523766 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-694cb6497d-45zn5"] Jun 15 03:16:38.526768 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:16:38.526744 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod277599cb_8940_479c_a249_5244193660d4.slice/crio-09ee757e4009eb0ac6736774e3b9ac648ef2ca57db9a174a61bc7b1d311618c2 WatchSource:0}: Error finding container 09ee757e4009eb0ac6736774e3b9ac648ef2ca57db9a174a61bc7b1d311618c2: Status 404 returned error can't find the container with id 09ee757e4009eb0ac6736774e3b9ac648ef2ca57db9a174a61bc7b1d311618c2 Jun 15 03:16:38.773184 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:38.773105 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-694cb6497d-45zn5" event={"ID":"277599cb-8940-479c-a249-5244193660d4","Type":"ContainerStarted","Data":"09ee757e4009eb0ac6736774e3b9ac648ef2ca57db9a174a61bc7b1d311618c2"} Jun 15 03:16:40.783462 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:40.783417 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-694cb6497d-45zn5" event={"ID":"277599cb-8940-479c-a249-5244193660d4","Type":"ContainerStarted","Data":"f0b4c10e3096e8362d7f5b383545150171ed0a9e92c01faa5258b4a343863909"} Jun 15 03:16:40.798715 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:40.798653 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-694cb6497d-45zn5" podStartSLOduration=0.696192289 podStartE2EDuration="2.798633312s" totalImagesPullingTime="2.102441023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:16:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:16:40.796957736 +0000 UTC m=+469.108096740" watchObservedRunningTime="2026-06-15 03:16:40.798633312 +0000 UTC m=+469.109772304" Jun 15 03:16:57.410426 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.410393 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft"] Jun 15 03:16:57.417566 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.417546 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.419793 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.419767 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-7hrcc\"" Jun 15 03:16:57.419916 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.419770 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 15 03:16:57.419972 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.419908 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 15 03:16:57.419972 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.419912 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 15 03:16:57.426802 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.426769 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft"] Jun 15 03:16:57.467303 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.467275 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0afa077d-107f-4f3f-a06d-455b8d8d0a15-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.467461 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.467310 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0afa077d-107f-4f3f-a06d-455b8d8d0a15-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.467461 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.467333 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hl9n\" (UniqueName: \"kubernetes.io/projected/0afa077d-107f-4f3f-a06d-455b8d8d0a15-kube-api-access-8hl9n\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.467461 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.467356 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0afa077d-107f-4f3f-a06d-455b8d8d0a15-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.467461 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.467414 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0afa077d-107f-4f3f-a06d-455b8d8d0a15-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.467628 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.467480 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0afa077d-107f-4f3f-a06d-455b8d8d0a15-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.467628 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.467511 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0afa077d-107f-4f3f-a06d-455b8d8d0a15-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.467628 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.467549 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0afa077d-107f-4f3f-a06d-455b8d8d0a15-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.467628 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.467570 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0afa077d-107f-4f3f-a06d-455b8d8d0a15-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.568248 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.568213 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0afa077d-107f-4f3f-a06d-455b8d8d0a15-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.568248 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.568248 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0afa077d-107f-4f3f-a06d-455b8d8d0a15-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.568564 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.568282 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0afa077d-107f-4f3f-a06d-455b8d8d0a15-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.568564 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.568307 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0afa077d-107f-4f3f-a06d-455b8d8d0a15-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.568564 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.568334 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0afa077d-107f-4f3f-a06d-455b8d8d0a15-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.568564 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.568367 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0afa077d-107f-4f3f-a06d-455b8d8d0a15-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.568564 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.568396 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hl9n\" (UniqueName: \"kubernetes.io/projected/0afa077d-107f-4f3f-a06d-455b8d8d0a15-kube-api-access-8hl9n\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.568564 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.568423 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0afa077d-107f-4f3f-a06d-455b8d8d0a15-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.568564 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.568454 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0afa077d-107f-4f3f-a06d-455b8d8d0a15-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.568972 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.568866 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0afa077d-107f-4f3f-a06d-455b8d8d0a15-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.568972 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.568908 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0afa077d-107f-4f3f-a06d-455b8d8d0a15-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.568972 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.568967 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0afa077d-107f-4f3f-a06d-455b8d8d0a15-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.569102 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.569020 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0afa077d-107f-4f3f-a06d-455b8d8d0a15-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.569102 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.569034 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0afa077d-107f-4f3f-a06d-455b8d8d0a15-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.570823 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.570804 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0afa077d-107f-4f3f-a06d-455b8d8d0a15-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.570909 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.570846 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0afa077d-107f-4f3f-a06d-455b8d8d0a15-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.576113 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.576085 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hl9n\" (UniqueName: \"kubernetes.io/projected/0afa077d-107f-4f3f-a06d-455b8d8d0a15-kube-api-access-8hl9n\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.576529 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.576512 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0afa077d-107f-4f3f-a06d-455b8d8d0a15-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795fk9ft\" (UID: \"0afa077d-107f-4f3f-a06d-455b8d8d0a15\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.728924 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.728820 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:16:57.864464 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:57.864434 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft"] Jun 15 03:16:57.866916 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:16:57.866887 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0afa077d_107f_4f3f_a06d_455b8d8d0a15.slice/crio-a3af3d5f4446d87a11f8249803b3fb5d359aa7053e86cca7cfd5f9b14471da84 WatchSource:0}: Error finding container a3af3d5f4446d87a11f8249803b3fb5d359aa7053e86cca7cfd5f9b14471da84: Status 404 returned error can't find the container with id a3af3d5f4446d87a11f8249803b3fb5d359aa7053e86cca7cfd5f9b14471da84 Jun 15 03:16:58.853616 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:16:58.853548 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" event={"ID":"0afa077d-107f-4f3f-a06d-455b8d8d0a15","Type":"ContainerStarted","Data":"a3af3d5f4446d87a11f8249803b3fb5d359aa7053e86cca7cfd5f9b14471da84"} Jun 15 03:17:00.537107 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:00.537057 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:17:00.537382 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:00.537155 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:17:00.537382 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:00.537191 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:17:00.864537 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:00.864497 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" event={"ID":"0afa077d-107f-4f3f-a06d-455b8d8d0a15","Type":"ContainerStarted","Data":"d20049b928ab3eb4d8c8eedd3965a03770938b1ec2e34bbb6e717969f50f9ffd"} Jun 15 03:17:00.883861 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:00.883804 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" podStartSLOduration=1.21601831 podStartE2EDuration="3.883788584s" totalImagesPullingTime="2.667770274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:17:00.881781685 +0000 UTC m=+489.192920674" watchObservedRunningTime="2026-06-15 03:17:00.883788584 +0000 UTC m=+489.194927573" Jun 15 03:17:01.729730 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:01.729689 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:17:01.734406 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:01.734380 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:17:01.868489 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:01.868464 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:17:01.869402 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:01.869385 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795fk9ft" Jun 15 03:17:17.174427 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:17.174388 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mdkb8"] Jun 15 03:17:17.186708 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:17.186676 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mdkb8"] Jun 15 03:17:17.186890 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:17.186806 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mdkb8" Jun 15 03:17:17.189633 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:17.189608 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 15 03:17:17.189633 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:17.189624 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 15 03:17:17.338185 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:17.338146 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqk6f\" (UniqueName: \"kubernetes.io/projected/bfefd56d-b8e7-40ec-9059-6857f6f915a3-kube-api-access-bqk6f\") pod \"kuadrant-operator-catalog-mdkb8\" (UID: \"bfefd56d-b8e7-40ec-9059-6857f6f915a3\") " pod="kuadrant-system/kuadrant-operator-catalog-mdkb8" Jun 15 03:17:17.439086 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:17.438997 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqk6f\" (UniqueName: \"kubernetes.io/projected/bfefd56d-b8e7-40ec-9059-6857f6f915a3-kube-api-access-bqk6f\") pod \"kuadrant-operator-catalog-mdkb8\" (UID: \"bfefd56d-b8e7-40ec-9059-6857f6f915a3\") " pod="kuadrant-system/kuadrant-operator-catalog-mdkb8" Jun 15 03:17:17.447197 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:17.447163 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqk6f\" (UniqueName: \"kubernetes.io/projected/bfefd56d-b8e7-40ec-9059-6857f6f915a3-kube-api-access-bqk6f\") pod \"kuadrant-operator-catalog-mdkb8\" (UID: \"bfefd56d-b8e7-40ec-9059-6857f6f915a3\") " pod="kuadrant-system/kuadrant-operator-catalog-mdkb8" Jun 15 03:17:17.497290 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:17.497251 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mdkb8" Jun 15 03:17:17.543326 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:17.543292 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mdkb8"] Jun 15 03:17:17.620710 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:17.620682 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mdkb8"] Jun 15 03:17:17.623281 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:17:17.623255 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfefd56d_b8e7_40ec_9059_6857f6f915a3.slice/crio-59a771a1cd23ae9be8bd1f55d8d1a9ed6d8e391d93e7c8a47ad482e8bcd6d7ce WatchSource:0}: Error finding container 59a771a1cd23ae9be8bd1f55d8d1a9ed6d8e391d93e7c8a47ad482e8bcd6d7ce: Status 404 returned error can't find the container with id 59a771a1cd23ae9be8bd1f55d8d1a9ed6d8e391d93e7c8a47ad482e8bcd6d7ce Jun 15 03:17:17.754476 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:17.754391 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-855sx"] Jun 15 03:17:17.762334 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:17.762171 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-855sx" Jun 15 03:17:17.764983 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:17.764956 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-8vndr\"" Jun 15 03:17:17.771128 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:17.770986 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-855sx"] Jun 15 03:17:17.841783 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:17.841744 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwspq\" (UniqueName: \"kubernetes.io/projected/32431758-698e-4d27-b975-2dade945af3a-kube-api-access-nwspq\") pod \"kuadrant-operator-catalog-855sx\" (UID: \"32431758-698e-4d27-b975-2dade945af3a\") " pod="kuadrant-system/kuadrant-operator-catalog-855sx" Jun 15 03:17:17.933914 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:17.933876 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mdkb8" event={"ID":"bfefd56d-b8e7-40ec-9059-6857f6f915a3","Type":"ContainerStarted","Data":"59a771a1cd23ae9be8bd1f55d8d1a9ed6d8e391d93e7c8a47ad482e8bcd6d7ce"} Jun 15 03:17:17.943239 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:17.943204 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwspq\" (UniqueName: \"kubernetes.io/projected/32431758-698e-4d27-b975-2dade945af3a-kube-api-access-nwspq\") pod \"kuadrant-operator-catalog-855sx\" (UID: \"32431758-698e-4d27-b975-2dade945af3a\") " pod="kuadrant-system/kuadrant-operator-catalog-855sx" Jun 15 03:17:17.954305 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:17.954278 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwspq\" (UniqueName: \"kubernetes.io/projected/32431758-698e-4d27-b975-2dade945af3a-kube-api-access-nwspq\") pod \"kuadrant-operator-catalog-855sx\" (UID: \"32431758-698e-4d27-b975-2dade945af3a\") " pod="kuadrant-system/kuadrant-operator-catalog-855sx" Jun 15 03:17:18.078613 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:18.078532 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-855sx" Jun 15 03:17:18.209272 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:18.209237 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-855sx"] Jun 15 03:17:18.212258 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:17:18.212225 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32431758_698e_4d27_b975_2dade945af3a.slice/crio-9768a17a35e6c8dea731bb034ddfc899cec3070e215a0c760e6c9db672efdd8c WatchSource:0}: Error finding container 9768a17a35e6c8dea731bb034ddfc899cec3070e215a0c760e6c9db672efdd8c: Status 404 returned error can't find the container with id 9768a17a35e6c8dea731bb034ddfc899cec3070e215a0c760e6c9db672efdd8c Jun 15 03:17:18.940306 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:18.940259 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-855sx" event={"ID":"32431758-698e-4d27-b975-2dade945af3a","Type":"ContainerStarted","Data":"9768a17a35e6c8dea731bb034ddfc899cec3070e215a0c760e6c9db672efdd8c"} Jun 15 03:17:19.946421 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:19.946382 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mdkb8" event={"ID":"bfefd56d-b8e7-40ec-9059-6857f6f915a3","Type":"ContainerStarted","Data":"0f01d8b8ac181af8f2d4ecbbd63cd89e27dcb95196d3b34c719ce4a6c2f83f6f"} Jun 15 03:17:19.946862 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:19.946443 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-mdkb8" podUID="bfefd56d-b8e7-40ec-9059-6857f6f915a3" containerName="registry-server" containerID="cri-o://0f01d8b8ac181af8f2d4ecbbd63cd89e27dcb95196d3b34c719ce4a6c2f83f6f" gracePeriod=2 Jun 15 03:17:19.947893 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:19.947871 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-855sx" event={"ID":"32431758-698e-4d27-b975-2dade945af3a","Type":"ContainerStarted","Data":"ad8f9ab4dca50f6ca1d219765ca443ea0e0ea5a1ee05d3584628c281b6f80431"} Jun 15 03:17:19.960777 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:19.960735 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-mdkb8" podStartSLOduration=0.854943023 podStartE2EDuration="2.960722405s" totalImagesPullingTime="2.105779382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:17:19.959320043 +0000 UTC m=+508.270459034" watchObservedRunningTime="2026-06-15 03:17:19.960722405 +0000 UTC m=+508.271861395" Jun 15 03:17:19.974794 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:19.974746 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-855sx" podStartSLOduration=1.438926329 podStartE2EDuration="2.974732772s" totalImagesPullingTime="1.535806443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:17:19.974160201 +0000 UTC m=+508.285299191" watchObservedRunningTime="2026-06-15 03:17:19.974732772 +0000 UTC m=+508.285871762" Jun 15 03:17:20.209003 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:20.208972 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mdkb8" Jun 15 03:17:20.367290 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:20.367256 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqk6f\" (UniqueName: \"kubernetes.io/projected/bfefd56d-b8e7-40ec-9059-6857f6f915a3-kube-api-access-bqk6f\") pod \"bfefd56d-b8e7-40ec-9059-6857f6f915a3\" (UID: \"bfefd56d-b8e7-40ec-9059-6857f6f915a3\") " Jun 15 03:17:20.369425 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:20.369401 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfefd56d-b8e7-40ec-9059-6857f6f915a3-kube-api-access-bqk6f" (OuterVolumeSpecName: "kube-api-access-bqk6f") pod "bfefd56d-b8e7-40ec-9059-6857f6f915a3" (UID: "bfefd56d-b8e7-40ec-9059-6857f6f915a3"). InnerVolumeSpecName "kube-api-access-bqk6f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:17:20.468463 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:20.468418 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqk6f\" (UniqueName: \"kubernetes.io/projected/bfefd56d-b8e7-40ec-9059-6857f6f915a3-kube-api-access-bqk6f\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:17:20.953249 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:20.953215 2562 generic.go:356] "Generic (PLEG): container finished" podID="bfefd56d-b8e7-40ec-9059-6857f6f915a3" containerID="0f01d8b8ac181af8f2d4ecbbd63cd89e27dcb95196d3b34c719ce4a6c2f83f6f" exitCode=0 Jun 15 03:17:20.953643 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:20.953257 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mdkb8" Jun 15 03:17:20.953643 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:20.953299 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mdkb8" event={"ID":"bfefd56d-b8e7-40ec-9059-6857f6f915a3","Type":"ContainerDied","Data":"0f01d8b8ac181af8f2d4ecbbd63cd89e27dcb95196d3b34c719ce4a6c2f83f6f"} Jun 15 03:17:20.953643 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:20.953369 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mdkb8" event={"ID":"bfefd56d-b8e7-40ec-9059-6857f6f915a3","Type":"ContainerDied","Data":"59a771a1cd23ae9be8bd1f55d8d1a9ed6d8e391d93e7c8a47ad482e8bcd6d7ce"} Jun 15 03:17:20.953643 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:20.953391 2562 scope.go:117] "RemoveContainer" containerID="0f01d8b8ac181af8f2d4ecbbd63cd89e27dcb95196d3b34c719ce4a6c2f83f6f" Jun 15 03:17:20.961767 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:20.961747 2562 scope.go:117] "RemoveContainer" containerID="0f01d8b8ac181af8f2d4ecbbd63cd89e27dcb95196d3b34c719ce4a6c2f83f6f" Jun 15 03:17:20.962022 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:17:20.962005 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f01d8b8ac181af8f2d4ecbbd63cd89e27dcb95196d3b34c719ce4a6c2f83f6f\": container with ID starting with 0f01d8b8ac181af8f2d4ecbbd63cd89e27dcb95196d3b34c719ce4a6c2f83f6f not found: ID does not exist" containerID="0f01d8b8ac181af8f2d4ecbbd63cd89e27dcb95196d3b34c719ce4a6c2f83f6f" Jun 15 03:17:20.962075 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:20.962029 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f01d8b8ac181af8f2d4ecbbd63cd89e27dcb95196d3b34c719ce4a6c2f83f6f"} err="failed to get container status \"0f01d8b8ac181af8f2d4ecbbd63cd89e27dcb95196d3b34c719ce4a6c2f83f6f\": rpc error: code = NotFound desc = could not find container \"0f01d8b8ac181af8f2d4ecbbd63cd89e27dcb95196d3b34c719ce4a6c2f83f6f\": container with ID starting with 0f01d8b8ac181af8f2d4ecbbd63cd89e27dcb95196d3b34c719ce4a6c2f83f6f not found: ID does not exist" Jun 15 03:17:20.971959 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:20.971938 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mdkb8"] Jun 15 03:17:20.977647 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:20.977617 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mdkb8"] Jun 15 03:17:22.044368 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:22.044334 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfefd56d-b8e7-40ec-9059-6857f6f915a3" path="/var/lib/kubelet/pods/bfefd56d-b8e7-40ec-9059-6857f6f915a3/volumes" Jun 15 03:17:28.078907 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:28.078864 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-855sx" Jun 15 03:17:28.079402 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:28.078931 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-855sx" Jun 15 03:17:28.100601 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:28.100573 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-855sx" Jun 15 03:17:29.007295 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:29.007266 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-855sx" Jun 15 03:17:49.787527 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:49.787447 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-vf427"] Jun 15 03:17:49.787929 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:49.787744 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfefd56d-b8e7-40ec-9059-6857f6f915a3" containerName="registry-server" Jun 15 03:17:49.787929 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:49.787755 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfefd56d-b8e7-40ec-9059-6857f6f915a3" containerName="registry-server" Jun 15 03:17:49.787929 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:49.787813 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfefd56d-b8e7-40ec-9059-6857f6f915a3" containerName="registry-server" Jun 15 03:17:49.793665 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:49.791502 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vf427" Jun 15 03:17:49.795688 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:49.795666 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-gcnbf\"" Jun 15 03:17:49.798919 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:49.798893 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-vf427"] Jun 15 03:17:49.919793 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:49.919758 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c42pr\" (UniqueName: \"kubernetes.io/projected/22016020-a0bf-418f-b15d-3c19de2410f5-kube-api-access-c42pr\") pod \"authorino-operator-75cc8d5cb7-vf427\" (UID: \"22016020-a0bf-418f-b15d-3c19de2410f5\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-vf427" Jun 15 03:17:50.020781 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:50.020735 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c42pr\" (UniqueName: \"kubernetes.io/projected/22016020-a0bf-418f-b15d-3c19de2410f5-kube-api-access-c42pr\") pod \"authorino-operator-75cc8d5cb7-vf427\" (UID: \"22016020-a0bf-418f-b15d-3c19de2410f5\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-vf427" Jun 15 03:17:50.034213 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:50.034183 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c42pr\" (UniqueName: \"kubernetes.io/projected/22016020-a0bf-418f-b15d-3c19de2410f5-kube-api-access-c42pr\") pod \"authorino-operator-75cc8d5cb7-vf427\" (UID: \"22016020-a0bf-418f-b15d-3c19de2410f5\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-vf427" Jun 15 03:17:50.103058 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:50.102978 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vf427" Jun 15 03:17:50.234978 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:50.234955 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-vf427"] Jun 15 03:17:50.237544 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:17:50.237516 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22016020_a0bf_418f_b15d_3c19de2410f5.slice/crio-c1e181f6798099ad2f2aa0377ad61a44bafe73b8428e66af3e4519e69cc238cd WatchSource:0}: Error finding container c1e181f6798099ad2f2aa0377ad61a44bafe73b8428e66af3e4519e69cc238cd: Status 404 returned error can't find the container with id c1e181f6798099ad2f2aa0377ad61a44bafe73b8428e66af3e4519e69cc238cd Jun 15 03:17:51.079651 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:51.079608 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vf427" event={"ID":"22016020-a0bf-418f-b15d-3c19de2410f5","Type":"ContainerStarted","Data":"c1e181f6798099ad2f2aa0377ad61a44bafe73b8428e66af3e4519e69cc238cd"} Jun 15 03:17:53.093865 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:53.093807 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vf427" event={"ID":"22016020-a0bf-418f-b15d-3c19de2410f5","Type":"ContainerStarted","Data":"f44f6f8da46669cdbc2f247a8053983dbcc17ef632cc0451e4095856b37be181"} Jun 15 03:17:53.094299 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:53.093895 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vf427" Jun 15 03:17:53.119365 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:53.119316 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vf427" podStartSLOduration=1.798489634 podStartE2EDuration="4.119301565s" totalImagesPullingTime="2.320811931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:17:53.117436352 +0000 UTC m=+541.428575354" watchObservedRunningTime="2026-06-15 03:17:53.119301565 +0000 UTC m=+541.430440555" Jun 15 03:17:54.930375 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:54.930342 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf"] Jun 15 03:17:54.933709 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:54.933689 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" Jun 15 03:17:54.937059 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:54.937033 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7f457\"" Jun 15 03:17:54.946740 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:54.946716 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf"] Jun 15 03:17:55.064820 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:55.064785 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mxtq\" (UniqueName: \"kubernetes.io/projected/50c6b20d-403e-4bae-86d3-de8cd5ff6677-kube-api-access-9mxtq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69knf\" (UID: \"50c6b20d-403e-4bae-86d3-de8cd5ff6677\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" Jun 15 03:17:55.065021 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:55.064849 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50c6b20d-403e-4bae-86d3-de8cd5ff6677-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69knf\" (UID: \"50c6b20d-403e-4bae-86d3-de8cd5ff6677\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" Jun 15 03:17:55.165386 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:55.165352 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mxtq\" (UniqueName: \"kubernetes.io/projected/50c6b20d-403e-4bae-86d3-de8cd5ff6677-kube-api-access-9mxtq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69knf\" (UID: \"50c6b20d-403e-4bae-86d3-de8cd5ff6677\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" Jun 15 03:17:55.165555 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:55.165391 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50c6b20d-403e-4bae-86d3-de8cd5ff6677-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69knf\" (UID: \"50c6b20d-403e-4bae-86d3-de8cd5ff6677\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" Jun 15 03:17:55.165771 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:55.165752 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50c6b20d-403e-4bae-86d3-de8cd5ff6677-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69knf\" (UID: \"50c6b20d-403e-4bae-86d3-de8cd5ff6677\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" Jun 15 03:17:55.175897 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:55.175870 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mxtq\" (UniqueName: \"kubernetes.io/projected/50c6b20d-403e-4bae-86d3-de8cd5ff6677-kube-api-access-9mxtq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69knf\" (UID: \"50c6b20d-403e-4bae-86d3-de8cd5ff6677\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" Jun 15 03:17:55.244000 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:55.243911 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" Jun 15 03:17:55.374931 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:55.374882 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf"] Jun 15 03:17:55.377649 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:17:55.377612 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50c6b20d_403e_4bae_86d3_de8cd5ff6677.slice/crio-3ce9d5976dc9e43948344cb5ce5acb824cb68535a1a7def10f2432c44b318936 WatchSource:0}: Error finding container 3ce9d5976dc9e43948344cb5ce5acb824cb68535a1a7def10f2432c44b318936: Status 404 returned error can't find the container with id 3ce9d5976dc9e43948344cb5ce5acb824cb68535a1a7def10f2432c44b318936 Jun 15 03:17:56.107030 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:17:56.106989 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" event={"ID":"50c6b20d-403e-4bae-86d3-de8cd5ff6677","Type":"ContainerStarted","Data":"3ce9d5976dc9e43948344cb5ce5acb824cb68535a1a7def10f2432c44b318936"} Jun 15 03:18:04.100522 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:04.100493 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vf427" Jun 15 03:18:04.145053 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:04.145013 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" event={"ID":"50c6b20d-403e-4bae-86d3-de8cd5ff6677","Type":"ContainerStarted","Data":"df2b47a1683525633562ad61fceae9783e27e5f880f376326297e64e4005aaec"} Jun 15 03:18:04.145208 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:04.145126 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" Jun 15 03:18:04.161899 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:04.161827 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" podStartSLOduration=1.5237524580000001 podStartE2EDuration="10.161809347s" totalImagesPullingTime="8.638056889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:18:04.161317196 +0000 UTC m=+552.472456187" watchObservedRunningTime="2026-06-15 03:18:04.161809347 +0000 UTC m=+552.472948336" Jun 15 03:18:15.152128 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:15.152088 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" Jun 15 03:18:16.219953 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.219919 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67"] Jun 15 03:18:16.223231 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.223208 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67" Jun 15 03:18:16.240686 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.240658 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67"] Jun 15 03:18:16.346201 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.346167 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zcwx\" (UniqueName: \"kubernetes.io/projected/68512b01-ba72-4926-ba51-1763213e60e0-kube-api-access-5zcwx\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8gs67\" (UID: \"68512b01-ba72-4926-ba51-1763213e60e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67" Jun 15 03:18:16.346374 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.346255 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/68512b01-ba72-4926-ba51-1763213e60e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8gs67\" (UID: \"68512b01-ba72-4926-ba51-1763213e60e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67" Jun 15 03:18:16.447365 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.447324 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/68512b01-ba72-4926-ba51-1763213e60e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8gs67\" (UID: \"68512b01-ba72-4926-ba51-1763213e60e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67" Jun 15 03:18:16.447556 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.447395 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zcwx\" (UniqueName: \"kubernetes.io/projected/68512b01-ba72-4926-ba51-1763213e60e0-kube-api-access-5zcwx\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8gs67\" (UID: \"68512b01-ba72-4926-ba51-1763213e60e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67" Jun 15 03:18:16.447750 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.447729 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/68512b01-ba72-4926-ba51-1763213e60e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8gs67\" (UID: \"68512b01-ba72-4926-ba51-1763213e60e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67" Jun 15 03:18:16.455030 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.455010 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zcwx\" (UniqueName: \"kubernetes.io/projected/68512b01-ba72-4926-ba51-1763213e60e0-kube-api-access-5zcwx\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8gs67\" (UID: \"68512b01-ba72-4926-ba51-1763213e60e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67" Jun 15 03:18:16.533276 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.533182 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67" Jun 15 03:18:16.668684 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.668659 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67"] Jun 15 03:18:16.671548 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:18:16.671506 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68512b01_ba72_4926_ba51_1763213e60e0.slice/crio-7dd4aa16c1f130c3aad3641679a0e5b6b16a2dcf454e60609635ebbf9f493d95 WatchSource:0}: Error finding container 7dd4aa16c1f130c3aad3641679a0e5b6b16a2dcf454e60609635ebbf9f493d95: Status 404 returned error can't find the container with id 7dd4aa16c1f130c3aad3641679a0e5b6b16a2dcf454e60609635ebbf9f493d95 Jun 15 03:18:16.825114 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.825020 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67"] Jun 15 03:18:16.830880 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.830851 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67"] Jun 15 03:18:16.852685 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.852652 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf"] Jun 15 03:18:16.852933 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.852905 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" podUID="50c6b20d-403e-4bae-86d3-de8cd5ff6677" containerName="manager" containerID="cri-o://df2b47a1683525633562ad61fceae9783e27e5f880f376326297e64e4005aaec" gracePeriod=2 Jun 15 03:18:16.856557 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.856526 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx"] Jun 15 03:18:16.856890 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.856874 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68512b01-ba72-4926-ba51-1763213e60e0" containerName="manager" Jun 15 03:18:16.856977 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.856893 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="68512b01-ba72-4926-ba51-1763213e60e0" containerName="manager" Jun 15 03:18:16.857034 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.857014 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="68512b01-ba72-4926-ba51-1763213e60e0" containerName="manager" Jun 15 03:18:16.860123 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.860101 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx" Jun 15 03:18:16.862749 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.862723 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf"] Jun 15 03:18:16.864339 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:18:16.864304 2562 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"50c6b20d-403e-4bae-86d3-de8cd5ff6677\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-69knf\": pods \"kuadrant-operator-controller-manager-8684c9ff49-69knf\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" Jun 15 03:18:16.866113 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:18:16.866087 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-69knf\" is forbidden: User \"system:node:ip-10-0-143-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-185.ec2.internal' and this object" podUID="50c6b20d-403e-4bae-86d3-de8cd5ff6677" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" Jun 15 03:18:16.876020 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.875939 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx"] Jun 15 03:18:16.887003 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.886970 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd"] Jun 15 03:18:16.887317 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.887303 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50c6b20d-403e-4bae-86d3-de8cd5ff6677" containerName="manager" Jun 15 03:18:16.887317 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.887318 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="50c6b20d-403e-4bae-86d3-de8cd5ff6677" containerName="manager" Jun 15 03:18:16.887422 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.887370 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="50c6b20d-403e-4bae-86d3-de8cd5ff6677" containerName="manager" Jun 15 03:18:16.890326 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.890307 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd" Jun 15 03:18:16.908489 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.908447 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd"] Jun 15 03:18:16.926900 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:18:16.926861 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-69knf\" is forbidden: User \"system:node:ip-10-0-143-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-185.ec2.internal' and this object" podUID="50c6b20d-403e-4bae-86d3-de8cd5ff6677" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" Jun 15 03:18:16.952275 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.952244 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b100628f-60f7-4be5-831a-9a72fb9a0ff4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-p8dzx\" (UID: \"b100628f-60f7-4be5-831a-9a72fb9a0ff4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx" Jun 15 03:18:16.952275 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:16.952278 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sqsk\" (UniqueName: \"kubernetes.io/projected/b100628f-60f7-4be5-831a-9a72fb9a0ff4-kube-api-access-5sqsk\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-p8dzx\" (UID: \"b100628f-60f7-4be5-831a-9a72fb9a0ff4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx" Jun 15 03:18:17.053536 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.053504 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b100628f-60f7-4be5-831a-9a72fb9a0ff4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-p8dzx\" (UID: \"b100628f-60f7-4be5-831a-9a72fb9a0ff4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx" Jun 15 03:18:17.053703 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.053543 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sqsk\" (UniqueName: \"kubernetes.io/projected/b100628f-60f7-4be5-831a-9a72fb9a0ff4-kube-api-access-5sqsk\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-p8dzx\" (UID: \"b100628f-60f7-4be5-831a-9a72fb9a0ff4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx" Jun 15 03:18:17.053703 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.053639 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ad14158-7b08-4f32-8520-152a967f957c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j5dgd\" (UID: \"8ad14158-7b08-4f32-8520-152a967f957c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd" Jun 15 03:18:17.053703 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.053671 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xpdv\" (UniqueName: \"kubernetes.io/projected/8ad14158-7b08-4f32-8520-152a967f957c-kube-api-access-5xpdv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j5dgd\" (UID: \"8ad14158-7b08-4f32-8520-152a967f957c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd" Jun 15 03:18:17.053946 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.053913 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b100628f-60f7-4be5-831a-9a72fb9a0ff4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-p8dzx\" (UID: \"b100628f-60f7-4be5-831a-9a72fb9a0ff4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx" Jun 15 03:18:17.065185 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.065159 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sqsk\" (UniqueName: \"kubernetes.io/projected/b100628f-60f7-4be5-831a-9a72fb9a0ff4-kube-api-access-5sqsk\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-p8dzx\" (UID: \"b100628f-60f7-4be5-831a-9a72fb9a0ff4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx" Jun 15 03:18:17.078786 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.078725 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" Jun 15 03:18:17.080993 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:18:17.080966 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-69knf\" is forbidden: User \"system:node:ip-10-0-143-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-185.ec2.internal' and this object" podUID="50c6b20d-403e-4bae-86d3-de8cd5ff6677" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" Jun 15 03:18:17.154858 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.154805 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ad14158-7b08-4f32-8520-152a967f957c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j5dgd\" (UID: \"8ad14158-7b08-4f32-8520-152a967f957c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd" Jun 15 03:18:17.154858 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.154856 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xpdv\" (UniqueName: \"kubernetes.io/projected/8ad14158-7b08-4f32-8520-152a967f957c-kube-api-access-5xpdv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j5dgd\" (UID: \"8ad14158-7b08-4f32-8520-152a967f957c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd" Jun 15 03:18:17.155200 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.155181 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ad14158-7b08-4f32-8520-152a967f957c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j5dgd\" (UID: \"8ad14158-7b08-4f32-8520-152a967f957c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd" Jun 15 03:18:17.170771 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.170729 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xpdv\" (UniqueName: \"kubernetes.io/projected/8ad14158-7b08-4f32-8520-152a967f957c-kube-api-access-5xpdv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-j5dgd\" (UID: \"8ad14158-7b08-4f32-8520-152a967f957c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd" Jun 15 03:18:17.207861 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.207805 2562 generic.go:356] "Generic (PLEG): container finished" podID="50c6b20d-403e-4bae-86d3-de8cd5ff6677" containerID="df2b47a1683525633562ad61fceae9783e27e5f880f376326297e64e4005aaec" exitCode=0 Jun 15 03:18:17.208025 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.207872 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" Jun 15 03:18:17.208025 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.207882 2562 scope.go:117] "RemoveContainer" containerID="df2b47a1683525633562ad61fceae9783e27e5f880f376326297e64e4005aaec" Jun 15 03:18:17.209684 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:18:17.209667 2562 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67" container="manager" Jun 15 03:18:17.209684 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:18:17.209675 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-69knf\" is forbidden: User \"system:node:ip-10-0-143-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-185.ec2.internal' and this object" podUID="50c6b20d-403e-4bae-86d3-de8cd5ff6677" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" Jun 15 03:18:17.211183 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:18:17.211156 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-69knf\" is forbidden: User \"system:node:ip-10-0-143-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-185.ec2.internal' and this object" podUID="50c6b20d-403e-4bae-86d3-de8cd5ff6677" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" Jun 15 03:18:17.212658 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.212640 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx" Jun 15 03:18:17.212774 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:18:17.212664 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8gs67\" is forbidden: User \"system:node:ip-10-0-143-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-185.ec2.internal' and this object" podUID="68512b01-ba72-4926-ba51-1763213e60e0" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67" Jun 15 03:18:17.217993 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.217862 2562 scope.go:117] "RemoveContainer" containerID="df2b47a1683525633562ad61fceae9783e27e5f880f376326297e64e4005aaec" Jun 15 03:18:17.218660 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:18:17.218522 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df2b47a1683525633562ad61fceae9783e27e5f880f376326297e64e4005aaec\": container with ID starting with df2b47a1683525633562ad61fceae9783e27e5f880f376326297e64e4005aaec not found: ID does not exist" containerID="df2b47a1683525633562ad61fceae9783e27e5f880f376326297e64e4005aaec" Jun 15 03:18:17.218660 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.218556 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df2b47a1683525633562ad61fceae9783e27e5f880f376326297e64e4005aaec"} err="failed to get container status \"df2b47a1683525633562ad61fceae9783e27e5f880f376326297e64e4005aaec\": rpc error: code = NotFound desc = could not find container \"df2b47a1683525633562ad61fceae9783e27e5f880f376326297e64e4005aaec\": container with ID starting with df2b47a1683525633562ad61fceae9783e27e5f880f376326297e64e4005aaec not found: ID does not exist" Jun 15 03:18:17.223361 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.223342 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd" Jun 15 03:18:17.255229 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.255193 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50c6b20d-403e-4bae-86d3-de8cd5ff6677-extensions-socket-volume\") pod \"50c6b20d-403e-4bae-86d3-de8cd5ff6677\" (UID: \"50c6b20d-403e-4bae-86d3-de8cd5ff6677\") " Jun 15 03:18:17.255419 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.255293 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mxtq\" (UniqueName: \"kubernetes.io/projected/50c6b20d-403e-4bae-86d3-de8cd5ff6677-kube-api-access-9mxtq\") pod \"50c6b20d-403e-4bae-86d3-de8cd5ff6677\" (UID: \"50c6b20d-403e-4bae-86d3-de8cd5ff6677\") " Jun 15 03:18:17.256019 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.255991 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50c6b20d-403e-4bae-86d3-de8cd5ff6677-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "50c6b20d-403e-4bae-86d3-de8cd5ff6677" (UID: "50c6b20d-403e-4bae-86d3-de8cd5ff6677"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:18:17.258382 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.258337 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50c6b20d-403e-4bae-86d3-de8cd5ff6677-kube-api-access-9mxtq" (OuterVolumeSpecName: "kube-api-access-9mxtq") pod "50c6b20d-403e-4bae-86d3-de8cd5ff6677" (UID: "50c6b20d-403e-4bae-86d3-de8cd5ff6677"). InnerVolumeSpecName "kube-api-access-9mxtq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:18:17.356321 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.356292 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mxtq\" (UniqueName: \"kubernetes.io/projected/50c6b20d-403e-4bae-86d3-de8cd5ff6677-kube-api-access-9mxtq\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:18:17.356483 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.356323 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/50c6b20d-403e-4bae-86d3-de8cd5ff6677-extensions-socket-volume\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:18:17.403689 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.403541 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx"] Jun 15 03:18:17.407065 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:17.405454 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd"] Jun 15 03:18:17.411711 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:18:17.411680 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ad14158_7b08_4f32_8520_152a967f957c.slice/crio-d37ad71e825e953676bf40a873db568f11bc66474064e38f605368357edca970 WatchSource:0}: Error finding container d37ad71e825e953676bf40a873db568f11bc66474064e38f605368357edca970: Status 404 returned error can't find the container with id d37ad71e825e953676bf40a873db568f11bc66474064e38f605368357edca970 Jun 15 03:18:17.518047 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:18:17.518006 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-69knf\" is forbidden: User \"system:node:ip-10-0-143-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-185.ec2.internal' and this object" podUID="50c6b20d-403e-4bae-86d3-de8cd5ff6677" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69knf" Jun 15 03:18:17.519937 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:18:17.519910 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8gs67\" is forbidden: User \"system:node:ip-10-0-143-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-185.ec2.internal' and this object" podUID="68512b01-ba72-4926-ba51-1763213e60e0" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67" Jun 15 03:18:18.045475 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:18.045438 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50c6b20d-403e-4bae-86d3-de8cd5ff6677" path="/var/lib/kubelet/pods/50c6b20d-403e-4bae-86d3-de8cd5ff6677/volumes" Jun 15 03:18:18.215814 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:18.215764 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67" podUID="68512b01-ba72-4926-ba51-1763213e60e0" containerName="manager" containerID="cri-o://f79c6065dfd90b35aca3e343dc43bcb22e8787e8a0b4942fbbaee78fa1e2afff" gracePeriod=2 Jun 15 03:18:18.218413 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:18.218384 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd" event={"ID":"8ad14158-7b08-4f32-8520-152a967f957c","Type":"ContainerStarted","Data":"42ad3b3ddecf98d1b572cfeaf3d67951225af16eafb80886bb7fefdbf9900739"} Jun 15 03:18:18.218573 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:18.218442 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd" event={"ID":"8ad14158-7b08-4f32-8520-152a967f957c","Type":"ContainerStarted","Data":"d37ad71e825e953676bf40a873db568f11bc66474064e38f605368357edca970"} Jun 15 03:18:18.218573 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:18.218474 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd" Jun 15 03:18:18.220380 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:18.220087 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx" event={"ID":"b100628f-60f7-4be5-831a-9a72fb9a0ff4","Type":"ContainerStarted","Data":"d9009661b38c8c73054a64aceb11f790b478f2cbd9476a0ebea7d4ca4136942c"} Jun 15 03:18:18.220380 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:18.220138 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx" Jun 15 03:18:18.220380 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:18.220162 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx" event={"ID":"b100628f-60f7-4be5-831a-9a72fb9a0ff4","Type":"ContainerStarted","Data":"34c730a0cfdb359bc57cada88984742e7fc7675a1e9fc8734fc58442d5e3f43a"} Jun 15 03:18:18.237956 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:18.237913 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd" podStartSLOduration=2.237894007 podStartE2EDuration="2.237894007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:18:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:18:18.23615056 +0000 UTC m=+566.547289549" watchObservedRunningTime="2026-06-15 03:18:18.237894007 +0000 UTC m=+566.549032998" Jun 15 03:18:18.252522 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:18.252473 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx" podStartSLOduration=2.252458328 podStartE2EDuration="2.252458328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:18:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:18:18.251812765 +0000 UTC m=+566.562951760" watchObservedRunningTime="2026-06-15 03:18:18.252458328 +0000 UTC m=+566.563597317" Jun 15 03:18:18.455963 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:18.455937 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67" Jun 15 03:18:18.457739 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:18:18.457715 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8gs67\" is forbidden: User \"system:node:ip-10-0-143-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-185.ec2.internal' and this object" podUID="68512b01-ba72-4926-ba51-1763213e60e0" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67" Jun 15 03:18:18.567419 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:18.567376 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zcwx\" (UniqueName: \"kubernetes.io/projected/68512b01-ba72-4926-ba51-1763213e60e0-kube-api-access-5zcwx\") pod \"68512b01-ba72-4926-ba51-1763213e60e0\" (UID: \"68512b01-ba72-4926-ba51-1763213e60e0\") " Jun 15 03:18:18.567588 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:18.567442 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/68512b01-ba72-4926-ba51-1763213e60e0-extensions-socket-volume\") pod \"68512b01-ba72-4926-ba51-1763213e60e0\" (UID: \"68512b01-ba72-4926-ba51-1763213e60e0\") " Jun 15 03:18:18.567662 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:18.567639 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68512b01-ba72-4926-ba51-1763213e60e0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "68512b01-ba72-4926-ba51-1763213e60e0" (UID: "68512b01-ba72-4926-ba51-1763213e60e0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:18:18.567730 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:18.567711 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/68512b01-ba72-4926-ba51-1763213e60e0-extensions-socket-volume\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:18:18.569483 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:18.569458 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68512b01-ba72-4926-ba51-1763213e60e0-kube-api-access-5zcwx" (OuterVolumeSpecName: "kube-api-access-5zcwx") pod "68512b01-ba72-4926-ba51-1763213e60e0" (UID: "68512b01-ba72-4926-ba51-1763213e60e0"). InnerVolumeSpecName "kube-api-access-5zcwx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:18:18.668170 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:18.668137 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zcwx\" (UniqueName: \"kubernetes.io/projected/68512b01-ba72-4926-ba51-1763213e60e0-kube-api-access-5zcwx\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:18:19.226237 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:19.226205 2562 generic.go:356] "Generic (PLEG): container finished" podID="68512b01-ba72-4926-ba51-1763213e60e0" containerID="f79c6065dfd90b35aca3e343dc43bcb22e8787e8a0b4942fbbaee78fa1e2afff" exitCode=2 Jun 15 03:18:19.226405 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:19.226251 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67" Jun 15 03:18:19.226405 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:19.226308 2562 scope.go:117] "RemoveContainer" containerID="f79c6065dfd90b35aca3e343dc43bcb22e8787e8a0b4942fbbaee78fa1e2afff" Jun 15 03:18:19.228389 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:18:19.228355 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8gs67\" is forbidden: User \"system:node:ip-10-0-143-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-185.ec2.internal' and this object" podUID="68512b01-ba72-4926-ba51-1763213e60e0" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67" Jun 15 03:18:19.234892 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:19.234876 2562 scope.go:117] "RemoveContainer" containerID="f79c6065dfd90b35aca3e343dc43bcb22e8787e8a0b4942fbbaee78fa1e2afff" Jun 15 03:18:19.235174 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:18:19.235153 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f79c6065dfd90b35aca3e343dc43bcb22e8787e8a0b4942fbbaee78fa1e2afff\": container with ID starting with f79c6065dfd90b35aca3e343dc43bcb22e8787e8a0b4942fbbaee78fa1e2afff not found: ID does not exist" containerID="f79c6065dfd90b35aca3e343dc43bcb22e8787e8a0b4942fbbaee78fa1e2afff" Jun 15 03:18:19.235272 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:19.235182 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f79c6065dfd90b35aca3e343dc43bcb22e8787e8a0b4942fbbaee78fa1e2afff"} err="failed to get container status \"f79c6065dfd90b35aca3e343dc43bcb22e8787e8a0b4942fbbaee78fa1e2afff\": rpc error: code = NotFound desc = could not find container \"f79c6065dfd90b35aca3e343dc43bcb22e8787e8a0b4942fbbaee78fa1e2afff\": container with ID starting with f79c6065dfd90b35aca3e343dc43bcb22e8787e8a0b4942fbbaee78fa1e2afff not found: ID does not exist" Jun 15 03:18:19.236059 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:18:19.236036 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-8gs67\" is forbidden: User \"system:node:ip-10-0-143-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-185.ec2.internal' and this object" podUID="68512b01-ba72-4926-ba51-1763213e60e0" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8gs67" Jun 15 03:18:20.044982 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:20.044949 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68512b01-ba72-4926-ba51-1763213e60e0" path="/var/lib/kubelet/pods/68512b01-ba72-4926-ba51-1763213e60e0/volumes" Jun 15 03:18:29.229320 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.229284 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx" Jun 15 03:18:29.229751 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.229356 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd" Jun 15 03:18:29.316154 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.316119 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd"] Jun 15 03:18:29.316369 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.316348 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd" podUID="8ad14158-7b08-4f32-8520-152a967f957c" containerName="manager" containerID="cri-o://42ad3b3ddecf98d1b572cfeaf3d67951225af16eafb80886bb7fefdbf9900739" gracePeriod=10 Jun 15 03:18:29.573940 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.573902 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd" Jun 15 03:18:29.582204 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.582176 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs"] Jun 15 03:18:29.582489 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.582477 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ad14158-7b08-4f32-8520-152a967f957c" containerName="manager" Jun 15 03:18:29.582529 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.582490 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ad14158-7b08-4f32-8520-152a967f957c" containerName="manager" Jun 15 03:18:29.582560 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.582548 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ad14158-7b08-4f32-8520-152a967f957c" containerName="manager" Jun 15 03:18:29.585548 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.585532 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs" Jun 15 03:18:29.600289 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.600262 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs"] Jun 15 03:18:29.655286 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.655249 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8880825f-2403-4ec2-ace8-2b34c681ef41-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4pqrs\" (UID: \"8880825f-2403-4ec2-ace8-2b34c681ef41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs" Jun 15 03:18:29.655455 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.655331 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddbnr\" (UniqueName: \"kubernetes.io/projected/8880825f-2403-4ec2-ace8-2b34c681ef41-kube-api-access-ddbnr\") pod \"kuadrant-operator-controller-manager-86878c769f-4pqrs\" (UID: \"8880825f-2403-4ec2-ace8-2b34c681ef41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs" Jun 15 03:18:29.756621 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.756475 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ad14158-7b08-4f32-8520-152a967f957c-extensions-socket-volume\") pod \"8ad14158-7b08-4f32-8520-152a967f957c\" (UID: \"8ad14158-7b08-4f32-8520-152a967f957c\") " Jun 15 03:18:29.756621 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.756603 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xpdv\" (UniqueName: \"kubernetes.io/projected/8ad14158-7b08-4f32-8520-152a967f957c-kube-api-access-5xpdv\") pod \"8ad14158-7b08-4f32-8520-152a967f957c\" (UID: \"8ad14158-7b08-4f32-8520-152a967f957c\") " Jun 15 03:18:29.756873 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.756806 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ad14158-7b08-4f32-8520-152a967f957c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8ad14158-7b08-4f32-8520-152a967f957c" (UID: "8ad14158-7b08-4f32-8520-152a967f957c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:18:29.756873 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.756817 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8880825f-2403-4ec2-ace8-2b34c681ef41-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4pqrs\" (UID: \"8880825f-2403-4ec2-ace8-2b34c681ef41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs" Jun 15 03:18:29.756983 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.756894 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddbnr\" (UniqueName: \"kubernetes.io/projected/8880825f-2403-4ec2-ace8-2b34c681ef41-kube-api-access-ddbnr\") pod \"kuadrant-operator-controller-manager-86878c769f-4pqrs\" (UID: \"8880825f-2403-4ec2-ace8-2b34c681ef41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs" Jun 15 03:18:29.757035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.757013 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8ad14158-7b08-4f32-8520-152a967f957c-extensions-socket-volume\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:18:29.757137 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.757121 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8880825f-2403-4ec2-ace8-2b34c681ef41-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4pqrs\" (UID: \"8880825f-2403-4ec2-ace8-2b34c681ef41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs" Jun 15 03:18:29.758714 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.758693 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ad14158-7b08-4f32-8520-152a967f957c-kube-api-access-5xpdv" (OuterVolumeSpecName: "kube-api-access-5xpdv") pod "8ad14158-7b08-4f32-8520-152a967f957c" (UID: "8ad14158-7b08-4f32-8520-152a967f957c"). InnerVolumeSpecName "kube-api-access-5xpdv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:18:29.764885 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.764861 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddbnr\" (UniqueName: \"kubernetes.io/projected/8880825f-2403-4ec2-ace8-2b34c681ef41-kube-api-access-ddbnr\") pod \"kuadrant-operator-controller-manager-86878c769f-4pqrs\" (UID: \"8880825f-2403-4ec2-ace8-2b34c681ef41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs" Jun 15 03:18:29.858167 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.858130 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xpdv\" (UniqueName: \"kubernetes.io/projected/8ad14158-7b08-4f32-8520-152a967f957c-kube-api-access-5xpdv\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:18:29.895081 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:29.895038 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs" Jun 15 03:18:30.030012 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:30.029992 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs"] Jun 15 03:18:30.032987 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:18:30.032957 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8880825f_2403_4ec2_ace8_2b34c681ef41.slice/crio-b3c2429bea838002b61584eb8363df77beaa55c67a63b7c45e13c9e1fffc77fe WatchSource:0}: Error finding container b3c2429bea838002b61584eb8363df77beaa55c67a63b7c45e13c9e1fffc77fe: Status 404 returned error can't find the container with id b3c2429bea838002b61584eb8363df77beaa55c67a63b7c45e13c9e1fffc77fe Jun 15 03:18:30.283111 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:30.283019 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs" event={"ID":"8880825f-2403-4ec2-ace8-2b34c681ef41","Type":"ContainerStarted","Data":"ef2ad480ea6031b261a9a1a97e2b627b3894693e019d9f939870c3fd6ca0de2e"} Jun 15 03:18:30.283111 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:30.283071 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs" event={"ID":"8880825f-2403-4ec2-ace8-2b34c681ef41","Type":"ContainerStarted","Data":"b3c2429bea838002b61584eb8363df77beaa55c67a63b7c45e13c9e1fffc77fe"} Jun 15 03:18:30.283531 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:30.283120 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs" Jun 15 03:18:30.284338 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:30.284318 2562 generic.go:356] "Generic (PLEG): container finished" podID="8ad14158-7b08-4f32-8520-152a967f957c" containerID="42ad3b3ddecf98d1b572cfeaf3d67951225af16eafb80886bb7fefdbf9900739" exitCode=0 Jun 15 03:18:30.284445 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:30.284355 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd" event={"ID":"8ad14158-7b08-4f32-8520-152a967f957c","Type":"ContainerDied","Data":"42ad3b3ddecf98d1b572cfeaf3d67951225af16eafb80886bb7fefdbf9900739"} Jun 15 03:18:30.284445 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:30.284382 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd" Jun 15 03:18:30.284445 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:30.284396 2562 scope.go:117] "RemoveContainer" containerID="42ad3b3ddecf98d1b572cfeaf3d67951225af16eafb80886bb7fefdbf9900739" Jun 15 03:18:30.284600 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:30.284386 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd" event={"ID":"8ad14158-7b08-4f32-8520-152a967f957c","Type":"ContainerDied","Data":"d37ad71e825e953676bf40a873db568f11bc66474064e38f605368357edca970"} Jun 15 03:18:30.292708 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:30.292691 2562 scope.go:117] "RemoveContainer" containerID="42ad3b3ddecf98d1b572cfeaf3d67951225af16eafb80886bb7fefdbf9900739" Jun 15 03:18:30.292952 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:18:30.292935 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42ad3b3ddecf98d1b572cfeaf3d67951225af16eafb80886bb7fefdbf9900739\": container with ID starting with 42ad3b3ddecf98d1b572cfeaf3d67951225af16eafb80886bb7fefdbf9900739 not found: ID does not exist" containerID="42ad3b3ddecf98d1b572cfeaf3d67951225af16eafb80886bb7fefdbf9900739" Jun 15 03:18:30.293009 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:30.292959 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42ad3b3ddecf98d1b572cfeaf3d67951225af16eafb80886bb7fefdbf9900739"} err="failed to get container status \"42ad3b3ddecf98d1b572cfeaf3d67951225af16eafb80886bb7fefdbf9900739\": rpc error: code = NotFound desc = could not find container \"42ad3b3ddecf98d1b572cfeaf3d67951225af16eafb80886bb7fefdbf9900739\": container with ID starting with 42ad3b3ddecf98d1b572cfeaf3d67951225af16eafb80886bb7fefdbf9900739 not found: ID does not exist" Jun 15 03:18:30.303724 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:30.303681 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs" podStartSLOduration=1.303666503 podStartE2EDuration="1.303666503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:18:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:18:30.301593308 +0000 UTC m=+578.612732300" watchObservedRunningTime="2026-06-15 03:18:30.303666503 +0000 UTC m=+578.614805492" Jun 15 03:18:30.318897 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:30.318871 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd"] Jun 15 03:18:30.325580 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:30.325557 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-j5dgd"] Jun 15 03:18:32.051225 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:32.051189 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ad14158-7b08-4f32-8520-152a967f957c" path="/var/lib/kubelet/pods/8ad14158-7b08-4f32-8520-152a967f957c/volumes" Jun 15 03:18:41.292041 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:41.292005 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs" Jun 15 03:18:41.343493 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:41.343451 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx"] Jun 15 03:18:41.343768 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:41.343713 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx" podUID="b100628f-60f7-4be5-831a-9a72fb9a0ff4" containerName="manager" containerID="cri-o://d9009661b38c8c73054a64aceb11f790b478f2cbd9476a0ebea7d4ca4136942c" gracePeriod=10 Jun 15 03:18:41.589485 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:41.589461 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx" Jun 15 03:18:41.666234 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:41.666200 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sqsk\" (UniqueName: \"kubernetes.io/projected/b100628f-60f7-4be5-831a-9a72fb9a0ff4-kube-api-access-5sqsk\") pod \"b100628f-60f7-4be5-831a-9a72fb9a0ff4\" (UID: \"b100628f-60f7-4be5-831a-9a72fb9a0ff4\") " Jun 15 03:18:41.666419 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:41.666311 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b100628f-60f7-4be5-831a-9a72fb9a0ff4-extensions-socket-volume\") pod \"b100628f-60f7-4be5-831a-9a72fb9a0ff4\" (UID: \"b100628f-60f7-4be5-831a-9a72fb9a0ff4\") " Jun 15 03:18:41.666679 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:41.666652 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b100628f-60f7-4be5-831a-9a72fb9a0ff4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b100628f-60f7-4be5-831a-9a72fb9a0ff4" (UID: "b100628f-60f7-4be5-831a-9a72fb9a0ff4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:18:41.668348 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:41.668323 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b100628f-60f7-4be5-831a-9a72fb9a0ff4-kube-api-access-5sqsk" (OuterVolumeSpecName: "kube-api-access-5sqsk") pod "b100628f-60f7-4be5-831a-9a72fb9a0ff4" (UID: "b100628f-60f7-4be5-831a-9a72fb9a0ff4"). InnerVolumeSpecName "kube-api-access-5sqsk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:18:41.767541 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:41.767499 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b100628f-60f7-4be5-831a-9a72fb9a0ff4-extensions-socket-volume\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:18:41.767541 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:41.767535 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sqsk\" (UniqueName: \"kubernetes.io/projected/b100628f-60f7-4be5-831a-9a72fb9a0ff4-kube-api-access-5sqsk\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:18:42.340266 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:42.340170 2562 generic.go:356] "Generic (PLEG): container finished" podID="b100628f-60f7-4be5-831a-9a72fb9a0ff4" containerID="d9009661b38c8c73054a64aceb11f790b478f2cbd9476a0ebea7d4ca4136942c" exitCode=0 Jun 15 03:18:42.340266 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:42.340230 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx" Jun 15 03:18:42.340266 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:42.340253 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx" event={"ID":"b100628f-60f7-4be5-831a-9a72fb9a0ff4","Type":"ContainerDied","Data":"d9009661b38c8c73054a64aceb11f790b478f2cbd9476a0ebea7d4ca4136942c"} Jun 15 03:18:42.340728 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:42.340310 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx" event={"ID":"b100628f-60f7-4be5-831a-9a72fb9a0ff4","Type":"ContainerDied","Data":"34c730a0cfdb359bc57cada88984742e7fc7675a1e9fc8734fc58442d5e3f43a"} Jun 15 03:18:42.340728 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:42.340333 2562 scope.go:117] "RemoveContainer" containerID="d9009661b38c8c73054a64aceb11f790b478f2cbd9476a0ebea7d4ca4136942c" Jun 15 03:18:42.349032 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:42.348824 2562 scope.go:117] "RemoveContainer" containerID="d9009661b38c8c73054a64aceb11f790b478f2cbd9476a0ebea7d4ca4136942c" Jun 15 03:18:42.349115 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:18:42.349094 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9009661b38c8c73054a64aceb11f790b478f2cbd9476a0ebea7d4ca4136942c\": container with ID starting with d9009661b38c8c73054a64aceb11f790b478f2cbd9476a0ebea7d4ca4136942c not found: ID does not exist" containerID="d9009661b38c8c73054a64aceb11f790b478f2cbd9476a0ebea7d4ca4136942c" Jun 15 03:18:42.349154 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:42.349124 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9009661b38c8c73054a64aceb11f790b478f2cbd9476a0ebea7d4ca4136942c"} err="failed to get container status \"d9009661b38c8c73054a64aceb11f790b478f2cbd9476a0ebea7d4ca4136942c\": rpc error: code = NotFound desc = could not find container \"d9009661b38c8c73054a64aceb11f790b478f2cbd9476a0ebea7d4ca4136942c\": container with ID starting with d9009661b38c8c73054a64aceb11f790b478f2cbd9476a0ebea7d4ca4136942c not found: ID does not exist" Jun 15 03:18:42.356200 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:42.356177 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx"] Jun 15 03:18:42.359927 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:42.359904 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-p8dzx"] Jun 15 03:18:44.045521 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:44.045487 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b100628f-60f7-4be5-831a-9a72fb9a0ff4" path="/var/lib/kubelet/pods/b100628f-60f7-4be5-831a-9a72fb9a0ff4/volumes" Jun 15 03:18:45.599544 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.599503 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z"] Jun 15 03:18:45.600030 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.600008 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b100628f-60f7-4be5-831a-9a72fb9a0ff4" containerName="manager" Jun 15 03:18:45.600100 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.600033 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b100628f-60f7-4be5-831a-9a72fb9a0ff4" containerName="manager" Jun 15 03:18:45.600151 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.600109 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b100628f-60f7-4be5-831a-9a72fb9a0ff4" containerName="manager" Jun 15 03:18:45.603210 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.603188 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.605027 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.605008 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-6r56j\"" Jun 15 03:18:45.614990 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.614964 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z"] Jun 15 03:18:45.703266 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.703225 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/363748fd-18d7-470c-9080-1bca3d60beec-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.703266 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.703262 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/363748fd-18d7-470c-9080-1bca3d60beec-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.703520 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.703303 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/363748fd-18d7-470c-9080-1bca3d60beec-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.703520 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.703326 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/363748fd-18d7-470c-9080-1bca3d60beec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.703520 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.703350 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/363748fd-18d7-470c-9080-1bca3d60beec-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.703520 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.703377 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/363748fd-18d7-470c-9080-1bca3d60beec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.703520 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.703394 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/363748fd-18d7-470c-9080-1bca3d60beec-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.703520 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.703458 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgs4n\" (UniqueName: \"kubernetes.io/projected/363748fd-18d7-470c-9080-1bca3d60beec-kube-api-access-sgs4n\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.703520 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.703502 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/363748fd-18d7-470c-9080-1bca3d60beec-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.804427 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.804377 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/363748fd-18d7-470c-9080-1bca3d60beec-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.804427 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.804435 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/363748fd-18d7-470c-9080-1bca3d60beec-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.804672 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.804458 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/363748fd-18d7-470c-9080-1bca3d60beec-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.804672 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.804496 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/363748fd-18d7-470c-9080-1bca3d60beec-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.804672 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.804515 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/363748fd-18d7-470c-9080-1bca3d60beec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.804672 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.804533 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/363748fd-18d7-470c-9080-1bca3d60beec-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.804672 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.804603 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/363748fd-18d7-470c-9080-1bca3d60beec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.804955 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.804859 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/363748fd-18d7-470c-9080-1bca3d60beec-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.804955 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.804896 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/363748fd-18d7-470c-9080-1bca3d60beec-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.805054 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.804652 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/363748fd-18d7-470c-9080-1bca3d60beec-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.805054 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.805039 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgs4n\" (UniqueName: \"kubernetes.io/projected/363748fd-18d7-470c-9080-1bca3d60beec-kube-api-access-sgs4n\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.805153 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.805123 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/363748fd-18d7-470c-9080-1bca3d60beec-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.805208 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.805193 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/363748fd-18d7-470c-9080-1bca3d60beec-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.805391 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.805369 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/363748fd-18d7-470c-9080-1bca3d60beec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.806927 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.806911 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/363748fd-18d7-470c-9080-1bca3d60beec-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.807146 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.807126 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/363748fd-18d7-470c-9080-1bca3d60beec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.812124 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.812103 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/363748fd-18d7-470c-9080-1bca3d60beec-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.812231 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.812208 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgs4n\" (UniqueName: \"kubernetes.io/projected/363748fd-18d7-470c-9080-1bca3d60beec-kube-api-access-sgs4n\") pod \"maas-default-gateway-openshift-default-687ff6996-5nd2z\" (UID: \"363748fd-18d7-470c-9080-1bca3d60beec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:45.916533 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:45.916500 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:46.079080 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:46.079051 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z"] Jun 15 03:18:46.081867 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:18:46.081800 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod363748fd_18d7_470c_9080_1bca3d60beec.slice/crio-20a2512a46de020616ce1cc8c6cfb9547a01fac8822ad94b2d21ce98c51ce0a1 WatchSource:0}: Error finding container 20a2512a46de020616ce1cc8c6cfb9547a01fac8822ad94b2d21ce98c51ce0a1: Status 404 returned error can't find the container with id 20a2512a46de020616ce1cc8c6cfb9547a01fac8822ad94b2d21ce98c51ce0a1 Jun 15 03:18:46.086904 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:46.086868 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:18:46.086990 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:46.086939 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:18:46.086990 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:46.086971 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:18:46.361190 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:46.361159 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" event={"ID":"363748fd-18d7-470c-9080-1bca3d60beec","Type":"ContainerStarted","Data":"431a5d400c43f67a8be2baafae544ea4fe036911eda1f9f2fde438a3ef18a7ab"} Jun 15 03:18:46.361381 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:46.361213 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" event={"ID":"363748fd-18d7-470c-9080-1bca3d60beec","Type":"ContainerStarted","Data":"20a2512a46de020616ce1cc8c6cfb9547a01fac8822ad94b2d21ce98c51ce0a1"} Jun 15 03:18:46.382094 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:46.382044 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" podStartSLOduration=1.382027419 podStartE2EDuration="1.382027419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:18:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:18:46.380211331 +0000 UTC m=+594.691350321" watchObservedRunningTime="2026-06-15 03:18:46.382027419 +0000 UTC m=+594.693166409" Jun 15 03:18:46.916806 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:46.916765 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:47.922357 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:47.922325 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:48.370706 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:48.370662 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:18:48.371731 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:18:48.371709 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:19:01.124959 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:19:01.124922 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:19:01.128236 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:19:01.128214 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-lcq9d" Jun 15 03:19:01.130238 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:19:01.130205 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 15 03:19:01.130353 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:19:01.130297 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bkprt\"" Jun 15 03:19:01.138124 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:19:01.138102 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:19:01.171198 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:19:01.171157 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:19:01.246742 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:19:01.246709 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a8f0d08c-5d9f-4a46-a402-aa7d027d0e8c-config-file\") pod \"limitador-limitador-54d6d988d5-lcq9d\" (UID: \"a8f0d08c-5d9f-4a46-a402-aa7d027d0e8c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-lcq9d" Jun 15 03:19:01.246946 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:19:01.246779 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6lrh\" (UniqueName: \"kubernetes.io/projected/a8f0d08c-5d9f-4a46-a402-aa7d027d0e8c-kube-api-access-f6lrh\") pod \"limitador-limitador-54d6d988d5-lcq9d\" (UID: \"a8f0d08c-5d9f-4a46-a402-aa7d027d0e8c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-lcq9d" Jun 15 03:19:01.347910 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:19:01.347869 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6lrh\" (UniqueName: \"kubernetes.io/projected/a8f0d08c-5d9f-4a46-a402-aa7d027d0e8c-kube-api-access-f6lrh\") pod \"limitador-limitador-54d6d988d5-lcq9d\" (UID: \"a8f0d08c-5d9f-4a46-a402-aa7d027d0e8c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-lcq9d" Jun 15 03:19:01.348072 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:19:01.347942 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a8f0d08c-5d9f-4a46-a402-aa7d027d0e8c-config-file\") pod \"limitador-limitador-54d6d988d5-lcq9d\" (UID: \"a8f0d08c-5d9f-4a46-a402-aa7d027d0e8c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-lcq9d" Jun 15 03:19:01.348493 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:19:01.348476 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a8f0d08c-5d9f-4a46-a402-aa7d027d0e8c-config-file\") pod \"limitador-limitador-54d6d988d5-lcq9d\" (UID: \"a8f0d08c-5d9f-4a46-a402-aa7d027d0e8c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-lcq9d" Jun 15 03:19:01.355443 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:19:01.355412 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6lrh\" (UniqueName: \"kubernetes.io/projected/a8f0d08c-5d9f-4a46-a402-aa7d027d0e8c-kube-api-access-f6lrh\") pod \"limitador-limitador-54d6d988d5-lcq9d\" (UID: \"a8f0d08c-5d9f-4a46-a402-aa7d027d0e8c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-lcq9d" Jun 15 03:19:01.438871 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:19:01.438754 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-lcq9d" Jun 15 03:19:01.568085 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:19:01.568059 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:19:01.570464 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:19:01.570437 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8f0d08c_5d9f_4a46_a402_aa7d027d0e8c.slice/crio-4b66b6df4838f72afa1788fad0d35449aac282ee157ac902ab2d4522ee863c15 WatchSource:0}: Error finding container 4b66b6df4838f72afa1788fad0d35449aac282ee157ac902ab2d4522ee863c15: Status 404 returned error can't find the container with id 4b66b6df4838f72afa1788fad0d35449aac282ee157ac902ab2d4522ee863c15 Jun 15 03:19:02.433887 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:19:02.433848 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-lcq9d" event={"ID":"a8f0d08c-5d9f-4a46-a402-aa7d027d0e8c","Type":"ContainerStarted","Data":"4b66b6df4838f72afa1788fad0d35449aac282ee157ac902ab2d4522ee863c15"} Jun 15 03:19:04.445071 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:19:04.445022 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-lcq9d" event={"ID":"a8f0d08c-5d9f-4a46-a402-aa7d027d0e8c","Type":"ContainerStarted","Data":"32af325540c7f1871a75fc7b4e7b6da32b9e23bcb753a7138d029a4a44a36b31"} Jun 15 03:19:04.445521 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:19:04.445102 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-lcq9d" Jun 15 03:19:04.460582 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:19:04.460521 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-lcq9d" podStartSLOduration=1.057660562 podStartE2EDuration="3.460504357s" totalImagesPullingTime="2.402843795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:19:04.459200541 +0000 UTC m=+612.770339559" watchObservedRunningTime="2026-06-15 03:19:04.460504357 +0000 UTC m=+612.771643350" Jun 15 03:19:15.450490 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:19:15.450407 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-lcq9d" Jun 15 03:21:08.702651 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:21:08.702553 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:21:17.026615 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:21:17.026579 2562 generic.go:356] "Generic (PLEG): container finished" podID="363748fd-18d7-470c-9080-1bca3d60beec" containerID="431a5d400c43f67a8be2baafae544ea4fe036911eda1f9f2fde438a3ef18a7ab" exitCode=0 Jun 15 03:21:17.027072 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:21:17.026654 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" event={"ID":"363748fd-18d7-470c-9080-1bca3d60beec","Type":"ContainerDied","Data":"431a5d400c43f67a8be2baafae544ea4fe036911eda1f9f2fde438a3ef18a7ab"} Jun 15 03:21:17.027119 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:21:17.027091 2562 scope.go:117] "RemoveContainer" containerID="431a5d400c43f67a8be2baafae544ea4fe036911eda1f9f2fde438a3ef18a7ab" Jun 15 03:21:17.028122 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:21:17.028092 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:21:17.028195 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:21:17.028175 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:21:17.028236 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:21:17.028223 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:21:18.033321 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:21:18.033282 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" event={"ID":"363748fd-18d7-470c-9080-1bca3d60beec","Type":"ContainerStarted","Data":"ce964e7ccd6e8ba70a7a5c9f8e211f14ce2ac2666612bd57be9ff8f0d61ee7ed"} Jun 15 03:21:18.371239 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:21:18.371207 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:21:18.917227 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:21:18.917189 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:21:18.918273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:21:18.918243 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-5nd2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jun 15 03:21:18.918393 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:21:18.918309 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" podUID="363748fd-18d7-470c-9080-1bca3d60beec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jun 15 03:21:19.917642 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:21:19.917605 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-5nd2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jun 15 03:21:19.918114 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:21:19.917662 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" podUID="363748fd-18d7-470c-9080-1bca3d60beec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jun 15 03:21:20.917852 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:21:20.917789 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-5nd2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jun 15 03:21:20.918223 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:21:20.917890 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" podUID="363748fd-18d7-470c-9080-1bca3d60beec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jun 15 03:21:21.920399 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:21:21.920368 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:21:21.921372 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:21:21.921353 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-5nd2z" Jun 15 03:22:04.441796 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:04.441756 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:22:11.590085 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.589995 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb"] Jun 15 03:22:11.593998 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.593968 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:11.596351 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.596325 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 15 03:22:11.597007 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.596982 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-nt4nn\"" Jun 15 03:22:11.597007 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.597005 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 15 03:22:11.597197 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.596993 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 15 03:22:11.604241 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.604211 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb"] Jun 15 03:22:11.747094 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.747061 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/169a22b8-615c-4c60-ae38-a0afca76134d-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-zb8mb\" (UID: \"169a22b8-615c-4c60-ae38-a0afca76134d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:11.747287 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.747117 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/169a22b8-615c-4c60-ae38-a0afca76134d-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-zb8mb\" (UID: \"169a22b8-615c-4c60-ae38-a0afca76134d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:11.747287 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.747134 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/169a22b8-615c-4c60-ae38-a0afca76134d-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-zb8mb\" (UID: \"169a22b8-615c-4c60-ae38-a0afca76134d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:11.747287 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.747230 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/169a22b8-615c-4c60-ae38-a0afca76134d-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-zb8mb\" (UID: \"169a22b8-615c-4c60-ae38-a0afca76134d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:11.747287 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.747261 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/169a22b8-615c-4c60-ae38-a0afca76134d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-zb8mb\" (UID: \"169a22b8-615c-4c60-ae38-a0afca76134d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:11.747509 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.747346 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwq2q\" (UniqueName: \"kubernetes.io/projected/169a22b8-615c-4c60-ae38-a0afca76134d-kube-api-access-bwq2q\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-zb8mb\" (UID: \"169a22b8-615c-4c60-ae38-a0afca76134d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:11.848010 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.847930 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/169a22b8-615c-4c60-ae38-a0afca76134d-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-zb8mb\" (UID: \"169a22b8-615c-4c60-ae38-a0afca76134d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:11.848010 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.847959 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/169a22b8-615c-4c60-ae38-a0afca76134d-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-zb8mb\" (UID: \"169a22b8-615c-4c60-ae38-a0afca76134d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:11.848010 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.848000 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/169a22b8-615c-4c60-ae38-a0afca76134d-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-zb8mb\" (UID: \"169a22b8-615c-4c60-ae38-a0afca76134d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:11.848273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.848116 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/169a22b8-615c-4c60-ae38-a0afca76134d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-zb8mb\" (UID: \"169a22b8-615c-4c60-ae38-a0afca76134d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:11.848273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.848173 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwq2q\" (UniqueName: \"kubernetes.io/projected/169a22b8-615c-4c60-ae38-a0afca76134d-kube-api-access-bwq2q\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-zb8mb\" (UID: \"169a22b8-615c-4c60-ae38-a0afca76134d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:11.848273 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.848200 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/169a22b8-615c-4c60-ae38-a0afca76134d-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-zb8mb\" (UID: \"169a22b8-615c-4c60-ae38-a0afca76134d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:11.848482 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.848460 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/169a22b8-615c-4c60-ae38-a0afca76134d-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-zb8mb\" (UID: \"169a22b8-615c-4c60-ae38-a0afca76134d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:11.848556 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.848504 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/169a22b8-615c-4c60-ae38-a0afca76134d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-zb8mb\" (UID: \"169a22b8-615c-4c60-ae38-a0afca76134d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:11.848556 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.848536 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/169a22b8-615c-4c60-ae38-a0afca76134d-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-zb8mb\" (UID: \"169a22b8-615c-4c60-ae38-a0afca76134d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:11.850282 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.850260 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/169a22b8-615c-4c60-ae38-a0afca76134d-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-zb8mb\" (UID: \"169a22b8-615c-4c60-ae38-a0afca76134d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:11.850534 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.850516 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/169a22b8-615c-4c60-ae38-a0afca76134d-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-zb8mb\" (UID: \"169a22b8-615c-4c60-ae38-a0afca76134d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:11.855463 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.855445 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwq2q\" (UniqueName: \"kubernetes.io/projected/169a22b8-615c-4c60-ae38-a0afca76134d-kube-api-access-bwq2q\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-zb8mb\" (UID: \"169a22b8-615c-4c60-ae38-a0afca76134d\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:11.905659 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.905623 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:11.927122 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:11.927093 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:22:12.035725 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:12.035629 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb"] Jun 15 03:22:12.038437 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:22:12.038405 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod169a22b8_615c_4c60_ae38_a0afca76134d.slice/crio-f0f1b88bf7a7ff447de714656b17b0f21b2465e93362298da49cb3e301e697b5 WatchSource:0}: Error finding container f0f1b88bf7a7ff447de714656b17b0f21b2465e93362298da49cb3e301e697b5: Status 404 returned error can't find the container with id f0f1b88bf7a7ff447de714656b17b0f21b2465e93362298da49cb3e301e697b5 Jun 15 03:22:12.040427 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:12.040407 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 15 03:22:12.273428 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:12.273393 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" event={"ID":"169a22b8-615c-4c60-ae38-a0afca76134d","Type":"ContainerStarted","Data":"f0f1b88bf7a7ff447de714656b17b0f21b2465e93362298da49cb3e301e697b5"} Jun 15 03:22:16.299374 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:16.299333 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" event={"ID":"169a22b8-615c-4c60-ae38-a0afca76134d","Type":"ContainerStarted","Data":"ff72ee06a9c0d57befceda291c704dda4e4bd84f5c726876c617ca9bdfb851d3"} Jun 15 03:22:17.306014 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:17.305967 2562 generic.go:356] "Generic (PLEG): container finished" podID="169a22b8-615c-4c60-ae38-a0afca76134d" containerID="ff72ee06a9c0d57befceda291c704dda4e4bd84f5c726876c617ca9bdfb851d3" exitCode=0 Jun 15 03:22:17.306482 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:17.306045 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" event={"ID":"169a22b8-615c-4c60-ae38-a0afca76134d","Type":"ContainerDied","Data":"ff72ee06a9c0d57befceda291c704dda4e4bd84f5c726876c617ca9bdfb851d3"} Jun 15 03:22:19.316393 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:19.316350 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" event={"ID":"169a22b8-615c-4c60-ae38-a0afca76134d","Type":"ContainerStarted","Data":"19b19d445748c43f7915f792eee6604fcc15a4891195187d410b88ebb555ad60"} Jun 15 03:22:19.333913 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:19.333860 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" podStartSLOduration=2.269631278 podStartE2EDuration="8.333827611s" totalImagesPullingTime="4.723898064s" totalInitContainerRuntime="1.340298269s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:22:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:22:19.332250603 +0000 UTC m=+807.643389598" watchObservedRunningTime="2026-06-15 03:22:19.333827611 +0000 UTC m=+807.644966601" Jun 15 03:22:21.744508 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:21.744470 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:22:21.906350 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:21.906317 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:21.906534 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:21.906404 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:21.919351 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:21.919322 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:22.341405 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:22.341371 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-zb8mb" Jun 15 03:22:36.612775 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.612738 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t"] Jun 15 03:22:36.635547 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.635519 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t"] Jun 15 03:22:36.635731 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.635655 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:22:36.637986 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.637965 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 15 03:22:36.770718 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.770679 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23d9d928-d3f5-42a5-b685-d4613e335d9e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t\" (UID: \"23d9d928-d3f5-42a5-b685-d4613e335d9e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:22:36.770718 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.770725 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23d9d928-d3f5-42a5-b685-d4613e335d9e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t\" (UID: \"23d9d928-d3f5-42a5-b685-d4613e335d9e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:22:36.771005 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.770791 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct8x9\" (UniqueName: \"kubernetes.io/projected/23d9d928-d3f5-42a5-b685-d4613e335d9e-kube-api-access-ct8x9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t\" (UID: \"23d9d928-d3f5-42a5-b685-d4613e335d9e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:22:36.771005 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.770866 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/23d9d928-d3f5-42a5-b685-d4613e335d9e-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t\" (UID: \"23d9d928-d3f5-42a5-b685-d4613e335d9e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:22:36.771005 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.770925 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23d9d928-d3f5-42a5-b685-d4613e335d9e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t\" (UID: \"23d9d928-d3f5-42a5-b685-d4613e335d9e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:22:36.771126 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.771017 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23d9d928-d3f5-42a5-b685-d4613e335d9e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t\" (UID: \"23d9d928-d3f5-42a5-b685-d4613e335d9e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:22:36.871908 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.871726 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct8x9\" (UniqueName: \"kubernetes.io/projected/23d9d928-d3f5-42a5-b685-d4613e335d9e-kube-api-access-ct8x9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t\" (UID: \"23d9d928-d3f5-42a5-b685-d4613e335d9e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:22:36.871908 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.871891 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/23d9d928-d3f5-42a5-b685-d4613e335d9e-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t\" (UID: \"23d9d928-d3f5-42a5-b685-d4613e335d9e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:22:36.872116 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.871928 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23d9d928-d3f5-42a5-b685-d4613e335d9e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t\" (UID: \"23d9d928-d3f5-42a5-b685-d4613e335d9e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:22:36.872116 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.871997 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23d9d928-d3f5-42a5-b685-d4613e335d9e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t\" (UID: \"23d9d928-d3f5-42a5-b685-d4613e335d9e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:22:36.872192 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.872134 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23d9d928-d3f5-42a5-b685-d4613e335d9e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t\" (UID: \"23d9d928-d3f5-42a5-b685-d4613e335d9e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:22:36.872192 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.872166 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23d9d928-d3f5-42a5-b685-d4613e335d9e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t\" (UID: \"23d9d928-d3f5-42a5-b685-d4613e335d9e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:22:36.872390 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.872366 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/23d9d928-d3f5-42a5-b685-d4613e335d9e-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t\" (UID: \"23d9d928-d3f5-42a5-b685-d4613e335d9e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:22:36.872456 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.872406 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23d9d928-d3f5-42a5-b685-d4613e335d9e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t\" (UID: \"23d9d928-d3f5-42a5-b685-d4613e335d9e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:22:36.872542 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.872524 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23d9d928-d3f5-42a5-b685-d4613e335d9e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t\" (UID: \"23d9d928-d3f5-42a5-b685-d4613e335d9e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:22:36.874550 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.874526 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23d9d928-d3f5-42a5-b685-d4613e335d9e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t\" (UID: \"23d9d928-d3f5-42a5-b685-d4613e335d9e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:22:36.874614 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.874580 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23d9d928-d3f5-42a5-b685-d4613e335d9e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t\" (UID: \"23d9d928-d3f5-42a5-b685-d4613e335d9e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:22:36.880931 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.880911 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct8x9\" (UniqueName: \"kubernetes.io/projected/23d9d928-d3f5-42a5-b685-d4613e335d9e-kube-api-access-ct8x9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t\" (UID: \"23d9d928-d3f5-42a5-b685-d4613e335d9e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:22:36.946495 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:36.946460 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:22:37.078921 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:37.078818 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t"] Jun 15 03:22:37.081070 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:22:37.081041 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23d9d928_d3f5_42a5_b685_d4613e335d9e.slice/crio-4401a6db190ebd3b58c54101ae6045ada2a4f2da81b6405a03fef310a519feac WatchSource:0}: Error finding container 4401a6db190ebd3b58c54101ae6045ada2a4f2da81b6405a03fef310a519feac: Status 404 returned error can't find the container with id 4401a6db190ebd3b58c54101ae6045ada2a4f2da81b6405a03fef310a519feac Jun 15 03:22:37.403150 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:37.403056 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" event={"ID":"23d9d928-d3f5-42a5-b685-d4613e335d9e","Type":"ContainerStarted","Data":"a42d8e52dd0e8ed36d59a284a21d9da6056509d92b14f2b3900fc49bfbbc5873"} Jun 15 03:22:37.403150 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:37.403115 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" event={"ID":"23d9d928-d3f5-42a5-b685-d4613e335d9e","Type":"ContainerStarted","Data":"4401a6db190ebd3b58c54101ae6045ada2a4f2da81b6405a03fef310a519feac"} Jun 15 03:22:37.536595 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:37.536556 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:22:44.130998 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:44.130964 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:22:50.214041 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.211148 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw"] Jun 15 03:22:50.216682 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.216641 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:22:50.220388 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.220361 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 15 03:22:50.224656 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.224630 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw"] Jun 15 03:22:50.297955 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.297920 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80456458-9f58-40fc-8c04-1779d22b34d3-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-25qdw\" (UID: \"80456458-9f58-40fc-8c04-1779d22b34d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:22:50.298140 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.297982 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmcz9\" (UniqueName: \"kubernetes.io/projected/80456458-9f58-40fc-8c04-1779d22b34d3-kube-api-access-zmcz9\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-25qdw\" (UID: \"80456458-9f58-40fc-8c04-1779d22b34d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:22:50.298140 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.298068 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/80456458-9f58-40fc-8c04-1779d22b34d3-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-25qdw\" (UID: \"80456458-9f58-40fc-8c04-1779d22b34d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:22:50.298140 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.298117 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80456458-9f58-40fc-8c04-1779d22b34d3-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-25qdw\" (UID: \"80456458-9f58-40fc-8c04-1779d22b34d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:22:50.298245 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.298145 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80456458-9f58-40fc-8c04-1779d22b34d3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-25qdw\" (UID: \"80456458-9f58-40fc-8c04-1779d22b34d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:22:50.298245 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.298169 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80456458-9f58-40fc-8c04-1779d22b34d3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-25qdw\" (UID: \"80456458-9f58-40fc-8c04-1779d22b34d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:22:50.399317 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.399277 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmcz9\" (UniqueName: \"kubernetes.io/projected/80456458-9f58-40fc-8c04-1779d22b34d3-kube-api-access-zmcz9\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-25qdw\" (UID: \"80456458-9f58-40fc-8c04-1779d22b34d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:22:50.399514 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.399346 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/80456458-9f58-40fc-8c04-1779d22b34d3-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-25qdw\" (UID: \"80456458-9f58-40fc-8c04-1779d22b34d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:22:50.399514 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.399390 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80456458-9f58-40fc-8c04-1779d22b34d3-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-25qdw\" (UID: \"80456458-9f58-40fc-8c04-1779d22b34d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:22:50.399514 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.399453 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80456458-9f58-40fc-8c04-1779d22b34d3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-25qdw\" (UID: \"80456458-9f58-40fc-8c04-1779d22b34d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:22:50.399514 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.399487 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80456458-9f58-40fc-8c04-1779d22b34d3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-25qdw\" (UID: \"80456458-9f58-40fc-8c04-1779d22b34d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:22:50.399719 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.399542 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80456458-9f58-40fc-8c04-1779d22b34d3-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-25qdw\" (UID: \"80456458-9f58-40fc-8c04-1779d22b34d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:22:50.399875 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.399846 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/80456458-9f58-40fc-8c04-1779d22b34d3-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-25qdw\" (UID: \"80456458-9f58-40fc-8c04-1779d22b34d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:22:50.399875 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.399859 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80456458-9f58-40fc-8c04-1779d22b34d3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-25qdw\" (UID: \"80456458-9f58-40fc-8c04-1779d22b34d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:22:50.400069 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.399933 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80456458-9f58-40fc-8c04-1779d22b34d3-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-25qdw\" (UID: \"80456458-9f58-40fc-8c04-1779d22b34d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:22:50.401874 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.401852 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80456458-9f58-40fc-8c04-1779d22b34d3-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-25qdw\" (UID: \"80456458-9f58-40fc-8c04-1779d22b34d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:22:50.402130 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.402112 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80456458-9f58-40fc-8c04-1779d22b34d3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-25qdw\" (UID: \"80456458-9f58-40fc-8c04-1779d22b34d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:22:50.406574 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.406551 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmcz9\" (UniqueName: \"kubernetes.io/projected/80456458-9f58-40fc-8c04-1779d22b34d3-kube-api-access-zmcz9\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-25qdw\" (UID: \"80456458-9f58-40fc-8c04-1779d22b34d3\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:22:50.530233 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.530139 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:22:50.663469 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:50.663437 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw"] Jun 15 03:22:50.666060 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:22:50.666032 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80456458_9f58_40fc_8c04_1779d22b34d3.slice/crio-10baea415f7c51e9ea4e73885de89d83e36cf7662a9ce5468247abb512202bde WatchSource:0}: Error finding container 10baea415f7c51e9ea4e73885de89d83e36cf7662a9ce5468247abb512202bde: Status 404 returned error can't find the container with id 10baea415f7c51e9ea4e73885de89d83e36cf7662a9ce5468247abb512202bde Jun 15 03:22:51.465200 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:51.465161 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" event={"ID":"80456458-9f58-40fc-8c04-1779d22b34d3","Type":"ContainerStarted","Data":"5c9aaf7a99ae85219f422017572fd55300ecb0875152920f92c6ef250a622022"} Jun 15 03:22:51.465200 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:51.465206 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" event={"ID":"80456458-9f58-40fc-8c04-1779d22b34d3","Type":"ContainerStarted","Data":"10baea415f7c51e9ea4e73885de89d83e36cf7662a9ce5468247abb512202bde"} Jun 15 03:22:51.533913 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:51.533871 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:22:52.470420 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:52.470389 2562 generic.go:356] "Generic (PLEG): container finished" podID="80456458-9f58-40fc-8c04-1779d22b34d3" containerID="5c9aaf7a99ae85219f422017572fd55300ecb0875152920f92c6ef250a622022" exitCode=0 Jun 15 03:22:52.470759 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:52.470468 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" event={"ID":"80456458-9f58-40fc-8c04-1779d22b34d3","Type":"ContainerDied","Data":"5c9aaf7a99ae85219f422017572fd55300ecb0875152920f92c6ef250a622022"} Jun 15 03:22:53.477303 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:53.477268 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" event={"ID":"80456458-9f58-40fc-8c04-1779d22b34d3","Type":"ContainerStarted","Data":"0ce009a41675a1d1c7fa27144b5db5d41f6c17de3fbd00406884bd81817a1cfb"} Jun 15 03:22:53.496573 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:22:53.496519 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" podStartSLOduration=1.7477143050000001 podStartE2EDuration="3.496503571s" totalImagesPullingTime="175.016494ms" totalInitContainerRuntime="1.573772772s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:22:53.494337098 +0000 UTC m=+841.805476101" watchObservedRunningTime="2026-06-15 03:22:53.496503571 +0000 UTC m=+841.807642560" Jun 15 03:23:00.530418 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:23:00.530385 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:23:00.530824 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:23:00.530665 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:23:00.542947 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:23:00.542926 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:23:01.526329 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:23:01.526295 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-25qdw" Jun 15 03:24:20.869538 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:24:20.869502 2562 generic.go:356] "Generic (PLEG): container finished" podID="23d9d928-d3f5-42a5-b685-d4613e335d9e" containerID="a42d8e52dd0e8ed36d59a284a21d9da6056509d92b14f2b3900fc49bfbbc5873" exitCode=0 Jun 15 03:24:20.869973 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:24:20.869578 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" event={"ID":"23d9d928-d3f5-42a5-b685-d4613e335d9e","Type":"ContainerDied","Data":"a42d8e52dd0e8ed36d59a284a21d9da6056509d92b14f2b3900fc49bfbbc5873"} Jun 15 03:24:21.876287 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:24:21.876246 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" event={"ID":"23d9d928-d3f5-42a5-b685-d4613e335d9e","Type":"ContainerStarted","Data":"d02d99a5a96fa0dbdc6ade45fe70f44e65824d14c8b9f90a8dbae49d444bd964"} Jun 15 03:24:21.895336 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:24:21.895288 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" podStartSLOduration=2.715439636 podStartE2EDuration="1m45.895274385s" totalImagesPullingTime="166.872022ms" totalInitContainerRuntime="1m43.012962727s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:22:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:24:21.893821564 +0000 UTC m=+930.204960566" watchObservedRunningTime="2026-06-15 03:24:21.895274385 +0000 UTC m=+930.206413375" Jun 15 03:24:26.946740 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:24:26.946681 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:24:26.947278 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:24:26.946785 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:24:26.959432 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:24:26.959402 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:24:27.914725 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:24:27.914692 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t" Jun 15 03:25:24.935395 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:25:24.935294 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:25:29.729711 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:25:29.729668 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:25:36.030900 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:25:36.030860 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:25:45.727854 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:25:45.727796 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:25:55.234040 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:25:55.234003 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:26:05.024742 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:26:05.024704 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:26:14.435972 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:26:14.435934 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:26:24.332646 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:26:24.332608 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:27:51.927241 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:27:51.927208 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:28:00.632923 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:28:00.632884 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:28:17.531312 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:28:17.531212 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:28:25.932056 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:28:25.932020 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:28:43.528857 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:28:43.528799 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:28:51.524766 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:28:51.524721 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:29:24.233262 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:29:24.233225 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:29:33.325440 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:29:33.325399 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:29:41.327571 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:29:41.327492 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:29:49.328154 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:29:49.328113 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:29:57.736251 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:29:57.736218 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:30:23.248929 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:30:23.248894 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:30:37.037348 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:30:37.037309 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:31:24.332337 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:31:24.332295 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:31:32.134013 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:31:32.133979 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:31:51.028457 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:31:51.028412 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:31:55.932799 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:31:55.932758 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:31:59.437586 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:31:59.437549 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:32:08.333209 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:32:08.333171 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:32:16.843078 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:32:16.843042 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:32:26.326880 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:32:26.326844 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:32:34.830607 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:32:34.830564 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:32:43.637084 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:32:43.637004 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:32:51.731949 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:32:51.731914 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:33:00.240762 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:33:00.240723 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:33:08.627442 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:33:08.627406 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:34:10.239102 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:34:10.239065 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs"] Jun 15 03:34:10.239658 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:34:10.239306 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs" podUID="8880825f-2403-4ec2-ace8-2b34c681ef41" containerName="manager" containerID="cri-o://ef2ad480ea6031b261a9a1a97e2b627b3894693e019d9f939870c3fd6ca0de2e" gracePeriod=10 Jun 15 03:34:10.483885 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:34:10.483860 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs" Jun 15 03:34:10.565080 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:34:10.564993 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8880825f-2403-4ec2-ace8-2b34c681ef41-extensions-socket-volume\") pod \"8880825f-2403-4ec2-ace8-2b34c681ef41\" (UID: \"8880825f-2403-4ec2-ace8-2b34c681ef41\") " Jun 15 03:34:10.565080 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:34:10.565031 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddbnr\" (UniqueName: \"kubernetes.io/projected/8880825f-2403-4ec2-ace8-2b34c681ef41-kube-api-access-ddbnr\") pod \"8880825f-2403-4ec2-ace8-2b34c681ef41\" (UID: \"8880825f-2403-4ec2-ace8-2b34c681ef41\") " Jun 15 03:34:10.565360 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:34:10.565334 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8880825f-2403-4ec2-ace8-2b34c681ef41-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8880825f-2403-4ec2-ace8-2b34c681ef41" (UID: "8880825f-2403-4ec2-ace8-2b34c681ef41"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:34:10.567148 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:34:10.567125 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8880825f-2403-4ec2-ace8-2b34c681ef41-kube-api-access-ddbnr" (OuterVolumeSpecName: "kube-api-access-ddbnr") pod "8880825f-2403-4ec2-ace8-2b34c681ef41" (UID: "8880825f-2403-4ec2-ace8-2b34c681ef41"). InnerVolumeSpecName "kube-api-access-ddbnr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:34:10.589431 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:34:10.589402 2562 generic.go:356] "Generic (PLEG): container finished" podID="8880825f-2403-4ec2-ace8-2b34c681ef41" containerID="ef2ad480ea6031b261a9a1a97e2b627b3894693e019d9f939870c3fd6ca0de2e" exitCode=0 Jun 15 03:34:10.589545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:34:10.589448 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs" Jun 15 03:34:10.589545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:34:10.589446 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs" event={"ID":"8880825f-2403-4ec2-ace8-2b34c681ef41","Type":"ContainerDied","Data":"ef2ad480ea6031b261a9a1a97e2b627b3894693e019d9f939870c3fd6ca0de2e"} Jun 15 03:34:10.589637 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:34:10.589577 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs" event={"ID":"8880825f-2403-4ec2-ace8-2b34c681ef41","Type":"ContainerDied","Data":"b3c2429bea838002b61584eb8363df77beaa55c67a63b7c45e13c9e1fffc77fe"} Jun 15 03:34:10.589637 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:34:10.589607 2562 scope.go:117] "RemoveContainer" containerID="ef2ad480ea6031b261a9a1a97e2b627b3894693e019d9f939870c3fd6ca0de2e" Jun 15 03:34:10.599155 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:34:10.599139 2562 scope.go:117] "RemoveContainer" containerID="ef2ad480ea6031b261a9a1a97e2b627b3894693e019d9f939870c3fd6ca0de2e" Jun 15 03:34:10.599413 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:34:10.599390 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef2ad480ea6031b261a9a1a97e2b627b3894693e019d9f939870c3fd6ca0de2e\": container with ID starting with ef2ad480ea6031b261a9a1a97e2b627b3894693e019d9f939870c3fd6ca0de2e not found: ID does not exist" containerID="ef2ad480ea6031b261a9a1a97e2b627b3894693e019d9f939870c3fd6ca0de2e" Jun 15 03:34:10.599461 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:34:10.599423 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef2ad480ea6031b261a9a1a97e2b627b3894693e019d9f939870c3fd6ca0de2e"} err="failed to get container status \"ef2ad480ea6031b261a9a1a97e2b627b3894693e019d9f939870c3fd6ca0de2e\": rpc error: code = NotFound desc = could not find container \"ef2ad480ea6031b261a9a1a97e2b627b3894693e019d9f939870c3fd6ca0de2e\": container with ID starting with ef2ad480ea6031b261a9a1a97e2b627b3894693e019d9f939870c3fd6ca0de2e not found: ID does not exist" Jun 15 03:34:10.611446 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:34:10.611422 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs"] Jun 15 03:34:10.615220 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:34:10.615195 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4pqrs"] Jun 15 03:34:10.665990 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:34:10.665957 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8880825f-2403-4ec2-ace8-2b34c681ef41-extensions-socket-volume\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:34:10.665990 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:34:10.665984 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddbnr\" (UniqueName: \"kubernetes.io/projected/8880825f-2403-4ec2-ace8-2b34c681ef41-kube-api-access-ddbnr\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:34:12.045404 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:34:12.045325 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8880825f-2403-4ec2-ace8-2b34c681ef41" path="/var/lib/kubelet/pods/8880825f-2403-4ec2-ace8-2b34c681ef41/volumes" Jun 15 03:35:11.334001 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:11.333962 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pkd5m"] Jun 15 03:35:11.334490 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:11.334332 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8880825f-2403-4ec2-ace8-2b34c681ef41" containerName="manager" Jun 15 03:35:11.334490 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:11.334343 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8880825f-2403-4ec2-ace8-2b34c681ef41" containerName="manager" Jun 15 03:35:11.334490 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:11.334402 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8880825f-2403-4ec2-ace8-2b34c681ef41" containerName="manager" Jun 15 03:35:11.338879 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:11.338862 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pkd5m" Jun 15 03:35:11.340875 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:11.340852 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7f457\"" Jun 15 03:35:11.350122 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:11.350097 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pkd5m"] Jun 15 03:35:11.421671 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:11.421627 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/886c9b2a-efc4-48c2-8210-64cc60cfc634-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pkd5m\" (UID: \"886c9b2a-efc4-48c2-8210-64cc60cfc634\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pkd5m" Jun 15 03:35:11.421671 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:11.421676 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgqwn\" (UniqueName: \"kubernetes.io/projected/886c9b2a-efc4-48c2-8210-64cc60cfc634-kube-api-access-qgqwn\") pod \"kuadrant-operator-controller-manager-86878c769f-pkd5m\" (UID: \"886c9b2a-efc4-48c2-8210-64cc60cfc634\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pkd5m" Jun 15 03:35:11.522756 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:11.522718 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/886c9b2a-efc4-48c2-8210-64cc60cfc634-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pkd5m\" (UID: \"886c9b2a-efc4-48c2-8210-64cc60cfc634\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pkd5m" Jun 15 03:35:11.522988 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:11.522763 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgqwn\" (UniqueName: \"kubernetes.io/projected/886c9b2a-efc4-48c2-8210-64cc60cfc634-kube-api-access-qgqwn\") pod \"kuadrant-operator-controller-manager-86878c769f-pkd5m\" (UID: \"886c9b2a-efc4-48c2-8210-64cc60cfc634\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pkd5m" Jun 15 03:35:11.523201 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:11.523176 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/886c9b2a-efc4-48c2-8210-64cc60cfc634-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pkd5m\" (UID: \"886c9b2a-efc4-48c2-8210-64cc60cfc634\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pkd5m" Jun 15 03:35:11.534532 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:11.534502 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgqwn\" (UniqueName: \"kubernetes.io/projected/886c9b2a-efc4-48c2-8210-64cc60cfc634-kube-api-access-qgqwn\") pod \"kuadrant-operator-controller-manager-86878c769f-pkd5m\" (UID: \"886c9b2a-efc4-48c2-8210-64cc60cfc634\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pkd5m" Jun 15 03:35:11.648545 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:11.648517 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pkd5m" Jun 15 03:35:11.788313 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:11.788265 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pkd5m"] Jun 15 03:35:11.790582 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:35:11.790554 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod886c9b2a_efc4_48c2_8210_64cc60cfc634.slice/crio-e1be745189c30ed0d48724b2ef0e5fe07938f4d581e9b967db6a047d4532614d WatchSource:0}: Error finding container e1be745189c30ed0d48724b2ef0e5fe07938f4d581e9b967db6a047d4532614d: Status 404 returned error can't find the container with id e1be745189c30ed0d48724b2ef0e5fe07938f4d581e9b967db6a047d4532614d Jun 15 03:35:11.872847 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:11.872807 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pkd5m" event={"ID":"886c9b2a-efc4-48c2-8210-64cc60cfc634","Type":"ContainerStarted","Data":"35f89e481e6f5a33facca76beec70632178863dfac423f334f714b5cf65a0b92"} Jun 15 03:35:11.872966 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:11.872879 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pkd5m" event={"ID":"886c9b2a-efc4-48c2-8210-64cc60cfc634","Type":"ContainerStarted","Data":"e1be745189c30ed0d48724b2ef0e5fe07938f4d581e9b967db6a047d4532614d"} Jun 15 03:35:11.872966 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:11.872903 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pkd5m" Jun 15 03:35:11.891989 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:11.891934 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pkd5m" podStartSLOduration=0.891918588 podStartE2EDuration="891.918588ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:35:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:35:11.890864976 +0000 UTC m=+1580.202003967" watchObservedRunningTime="2026-06-15 03:35:11.891918588 +0000 UTC m=+1580.203057577" Jun 15 03:35:22.879110 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:22.879072 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pkd5m" Jun 15 03:35:30.132799 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:30.132757 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:35:34.429122 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:35:34.429088 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:36:00.533941 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:36:00.533857 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:36:04.033681 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:36:04.033647 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:36:14.633472 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:36:14.633430 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:36:23.326358 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:36:23.326319 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:36:34.126981 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:36:34.126934 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:36:43.028305 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:36:43.028265 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:36:53.637694 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:36:53.637631 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:37:03.034673 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:37:03.034633 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:37:12.725446 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:37:12.725368 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:37:22.031877 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:37:22.031824 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:37:31.138985 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:37:31.138948 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:37:36.234847 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:37:36.234800 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:38:03.832976 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:38:03.832941 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:38:48.035692 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:38:48.035645 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:38:54.435950 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:38:54.435908 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:39:05.135109 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:39:05.135074 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:39:12.926919 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:39:12.926878 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:39:22.231946 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:39:22.231911 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:39:34.035224 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:39:34.035183 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:39:43.632851 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:39:43.632809 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:39:47.530555 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:39:47.530518 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:39:57.933208 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:39:57.933173 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:40:05.431958 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:40:05.431920 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:40:13.431575 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:40:13.431482 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:40:24.131384 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:40:24.131347 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:40:42.775214 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:40:42.775172 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:40:50.577920 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:40:50.577885 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:41:00.130094 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:41:00.130058 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:41:07.432543 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:41:07.432508 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:41:25.636978 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:41:25.636942 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:41:32.224633 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:41:32.224593 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:41:42.836277 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:41:42.836182 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:41:50.129962 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:41:50.129926 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:42:00.129055 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:42:00.129021 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:42:07.934512 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:42:07.934475 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:42:17.429186 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:42:17.429145 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:42:29.338303 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:42:29.338263 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:42:39.833906 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:42:39.833864 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:42:51.829088 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:42:51.829049 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:43:01.936946 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:43:01.934176 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:43:05.933349 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:43:05.933309 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:43:16.538344 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:43:16.538262 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:43:20.524786 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:43:20.524751 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:43:37.232459 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:43:37.232428 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:43:46.030252 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:43:46.030214 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:43:55.328875 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:43:55.328820 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:44:01.527540 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:01.527503 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:44:22.733659 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:22.733624 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:44:34.736987 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:34.736948 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:44:35.950780 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:35.950744 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4"] Jun 15 03:44:35.954460 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:35.954442 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:35.956468 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:35.956444 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-16200740-openshift-default-dockercfg-t7lkh\"" Jun 15 03:44:35.974543 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:35.974518 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4"] Jun 15 03:44:36.031581 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.031555 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-workload-certs\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.031691 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.031584 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-istio-envoy\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.031691 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.031604 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17785418-7616-4271-9e81-f7993dd0b8e2-istiod-ca-cert\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.031691 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.031661 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-istio-data\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.031868 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.031703 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-workload-socket\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.031868 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.031723 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6rvx\" (UniqueName: \"kubernetes.io/projected/17785418-7616-4271-9e81-f7993dd0b8e2-kube-api-access-q6rvx\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.031868 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.031749 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17785418-7616-4271-9e81-f7993dd0b8e2-istio-token\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.031868 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.031798 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-credential-socket\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.032050 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.031882 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17785418-7616-4271-9e81-f7993dd0b8e2-istio-podinfo\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.133188 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.133152 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-credential-socket\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.133350 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.133205 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17785418-7616-4271-9e81-f7993dd0b8e2-istio-podinfo\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.133350 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.133263 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-workload-certs\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.133350 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.133290 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-istio-envoy\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.133350 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.133317 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17785418-7616-4271-9e81-f7993dd0b8e2-istiod-ca-cert\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.133597 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.133370 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-istio-data\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.133597 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.133404 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-workload-socket\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.133597 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.133429 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6rvx\" (UniqueName: \"kubernetes.io/projected/17785418-7616-4271-9e81-f7993dd0b8e2-kube-api-access-q6rvx\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.133597 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.133471 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17785418-7616-4271-9e81-f7993dd0b8e2-istio-token\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.133798 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.133634 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-credential-socket\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.133798 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.133662 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-workload-certs\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.134046 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.134023 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-istio-data\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.134125 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.134067 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-workload-socket\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.134165 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.134147 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17785418-7616-4271-9e81-f7993dd0b8e2-istiod-ca-cert\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.135673 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.135653 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-istio-envoy\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.136044 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.136022 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17785418-7616-4271-9e81-f7993dd0b8e2-istio-podinfo\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.142464 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.142436 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17785418-7616-4271-9e81-f7993dd0b8e2-istio-token\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.142550 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.142477 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6rvx\" (UniqueName: \"kubernetes.io/projected/17785418-7616-4271-9e81-f7993dd0b8e2-kube-api-access-q6rvx\") pod \"e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.265185 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.265093 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:36.399517 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.399492 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4"] Jun 15 03:44:36.402200 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:44:36.402172 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17785418_7616_4271_9e81_f7993dd0b8e2.slice/crio-ef4077108e7f6447b0421e4a0cdb4e92b8e822c5a50f65309a4bc2b79e79007f WatchSource:0}: Error finding container ef4077108e7f6447b0421e4a0cdb4e92b8e822c5a50f65309a4bc2b79e79007f: Status 404 returned error can't find the container with id ef4077108e7f6447b0421e4a0cdb4e92b8e822c5a50f65309a4bc2b79e79007f Jun 15 03:44:36.404494 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.404457 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:44:36.404579 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.404524 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:44:36.404579 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.404558 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:44:36.506758 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.506716 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" event={"ID":"17785418-7616-4271-9e81-f7993dd0b8e2","Type":"ContainerStarted","Data":"f55ceca601ac5a604234027717efe948a856347501d9728bfed71776c4e7fc1e"} Jun 15 03:44:36.506974 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:36.506786 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" event={"ID":"17785418-7616-4271-9e81-f7993dd0b8e2","Type":"ContainerStarted","Data":"ef4077108e7f6447b0421e4a0cdb4e92b8e822c5a50f65309a4bc2b79e79007f"} Jun 15 03:44:37.266057 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:37.266019 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:37.270907 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:37.270874 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:37.294324 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:37.294276 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" podStartSLOduration=2.294261265 podStartE2EDuration="2.294261265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:44:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:44:36.529175615 +0000 UTC m=+2144.840314606" watchObservedRunningTime="2026-06-15 03:44:37.294261265 +0000 UTC m=+2145.605400255" Jun 15 03:44:37.511709 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:37.511680 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:37.512642 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:37.512619 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:40.730221 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:40.730181 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:44:42.344702 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:42.344666 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4"] Jun 15 03:44:42.345171 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:42.344850 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" podUID="17785418-7616-4271-9e81-f7993dd0b8e2" containerName="istio-proxy" containerID="cri-o://f55ceca601ac5a604234027717efe948a856347501d9728bfed71776c4e7fc1e" gracePeriod=30 Jun 15 03:44:47.560517 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.560482 2562 generic.go:356] "Generic (PLEG): container finished" podID="17785418-7616-4271-9e81-f7993dd0b8e2" containerID="f55ceca601ac5a604234027717efe948a856347501d9728bfed71776c4e7fc1e" exitCode=0 Jun 15 03:44:47.561000 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.560547 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" event={"ID":"17785418-7616-4271-9e81-f7993dd0b8e2","Type":"ContainerDied","Data":"f55ceca601ac5a604234027717efe948a856347501d9728bfed71776c4e7fc1e"} Jun 15 03:44:47.595370 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.595342 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:47.743728 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.743699 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-credential-socket\") pod \"17785418-7616-4271-9e81-f7993dd0b8e2\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " Jun 15 03:44:47.743946 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.743741 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17785418-7616-4271-9e81-f7993dd0b8e2-istio-token\") pod \"17785418-7616-4271-9e81-f7993dd0b8e2\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " Jun 15 03:44:47.743946 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.743772 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-workload-certs\") pod \"17785418-7616-4271-9e81-f7993dd0b8e2\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " Jun 15 03:44:47.743946 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.743796 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-istio-envoy\") pod \"17785418-7616-4271-9e81-f7993dd0b8e2\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " Jun 15 03:44:47.743946 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.743820 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-istio-data\") pod \"17785418-7616-4271-9e81-f7993dd0b8e2\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " Jun 15 03:44:47.743946 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.743866 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-workload-socket\") pod \"17785418-7616-4271-9e81-f7993dd0b8e2\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " Jun 15 03:44:47.743946 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.743895 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17785418-7616-4271-9e81-f7993dd0b8e2-istio-podinfo\") pod \"17785418-7616-4271-9e81-f7993dd0b8e2\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " Jun 15 03:44:47.743946 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.743939 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6rvx\" (UniqueName: \"kubernetes.io/projected/17785418-7616-4271-9e81-f7993dd0b8e2-kube-api-access-q6rvx\") pod \"17785418-7616-4271-9e81-f7993dd0b8e2\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " Jun 15 03:44:47.744315 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.743992 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17785418-7616-4271-9e81-f7993dd0b8e2-istiod-ca-cert\") pod \"17785418-7616-4271-9e81-f7993dd0b8e2\" (UID: \"17785418-7616-4271-9e81-f7993dd0b8e2\") " Jun 15 03:44:47.744315 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.744054 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "17785418-7616-4271-9e81-f7993dd0b8e2" (UID: "17785418-7616-4271-9e81-f7993dd0b8e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:44:47.744315 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.744174 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "17785418-7616-4271-9e81-f7993dd0b8e2" (UID: "17785418-7616-4271-9e81-f7993dd0b8e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:44:47.744315 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.744219 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-istio-data" (OuterVolumeSpecName: "istio-data") pod "17785418-7616-4271-9e81-f7993dd0b8e2" (UID: "17785418-7616-4271-9e81-f7993dd0b8e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:44:47.744476 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.744345 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-credential-socket\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:44:47.744476 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.744368 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-istio-data\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:44:47.744476 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.744384 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-workload-socket\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:44:47.744636 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.744458 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17785418-7616-4271-9e81-f7993dd0b8e2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "17785418-7616-4271-9e81-f7993dd0b8e2" (UID: "17785418-7616-4271-9e81-f7993dd0b8e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:44:47.744636 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.744527 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "17785418-7616-4271-9e81-f7993dd0b8e2" (UID: "17785418-7616-4271-9e81-f7993dd0b8e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:44:47.746180 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.746153 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17785418-7616-4271-9e81-f7993dd0b8e2-istio-token" (OuterVolumeSpecName: "istio-token") pod "17785418-7616-4271-9e81-f7993dd0b8e2" (UID: "17785418-7616-4271-9e81-f7993dd0b8e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:44:47.746269 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.746248 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/17785418-7616-4271-9e81-f7993dd0b8e2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "17785418-7616-4271-9e81-f7993dd0b8e2" (UID: "17785418-7616-4271-9e81-f7993dd0b8e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 15 03:44:47.746676 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.746647 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "17785418-7616-4271-9e81-f7993dd0b8e2" (UID: "17785418-7616-4271-9e81-f7993dd0b8e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:44:47.746771 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.746653 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17785418-7616-4271-9e81-f7993dd0b8e2-kube-api-access-q6rvx" (OuterVolumeSpecName: "kube-api-access-q6rvx") pod "17785418-7616-4271-9e81-f7993dd0b8e2" (UID: "17785418-7616-4271-9e81-f7993dd0b8e2"). InnerVolumeSpecName "kube-api-access-q6rvx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:44:47.845806 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.845777 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17785418-7616-4271-9e81-f7993dd0b8e2-istiod-ca-cert\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:44:47.845806 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.845802 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17785418-7616-4271-9e81-f7993dd0b8e2-istio-token\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:44:47.845806 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.845810 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-workload-certs\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:44:47.846033 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.845818 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17785418-7616-4271-9e81-f7993dd0b8e2-istio-envoy\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:44:47.846033 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.845826 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17785418-7616-4271-9e81-f7993dd0b8e2-istio-podinfo\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:44:47.846033 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:47.845851 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6rvx\" (UniqueName: \"kubernetes.io/projected/17785418-7616-4271-9e81-f7993dd0b8e2-kube-api-access-q6rvx\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:44:48.456399 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.456352 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52"] Jun 15 03:44:48.457092 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.457058 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17785418-7616-4271-9e81-f7993dd0b8e2" containerName="istio-proxy" Jun 15 03:44:48.457092 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.457091 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="17785418-7616-4271-9e81-f7993dd0b8e2" containerName="istio-proxy" Jun 15 03:44:48.457304 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.457187 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="17785418-7616-4271-9e81-f7993dd0b8e2" containerName="istio-proxy" Jun 15 03:44:48.467493 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.467465 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.469518 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.469497 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-554b1842-openshift-default-dockercfg-bszmm\"" Jun 15 03:44:48.474019 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.473994 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52"] Jun 15 03:44:48.552884 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.552827 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-token\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.552884 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.552892 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-credential-socket\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.553105 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.552922 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-workload-certs\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.553105 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.552942 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w8kb\" (UniqueName: \"kubernetes.io/projected/332af4f4-dafa-41e4-b836-05917a3fc1b4-kube-api-access-9w8kb\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.553105 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.553053 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-workload-socket\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.553105 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.553083 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/332af4f4-dafa-41e4-b836-05917a3fc1b4-istiod-ca-cert\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.553105 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.553106 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-envoy\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.553300 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.553122 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-data\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.553300 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.553160 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-podinfo\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.567447 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.567408 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" event={"ID":"17785418-7616-4271-9e81-f7993dd0b8e2","Type":"ContainerDied","Data":"ef4077108e7f6447b0421e4a0cdb4e92b8e822c5a50f65309a4bc2b79e79007f"} Jun 15 03:44:48.567916 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.567486 2562 scope.go:117] "RemoveContainer" containerID="f55ceca601ac5a604234027717efe948a856347501d9728bfed71776c4e7fc1e" Jun 15 03:44:48.567916 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.567492 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4" Jun 15 03:44:48.587332 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.587303 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4"] Jun 15 03:44:48.591893 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.591872 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-16200740-openshift-default-7b696fd5bb-zjkq4"] Jun 15 03:44:48.654604 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.654572 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-workload-socket\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.654604 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.654606 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/332af4f4-dafa-41e4-b836-05917a3fc1b4-istiod-ca-cert\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.654983 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.654631 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-envoy\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.654983 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.654752 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-data\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.654983 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.654857 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-podinfo\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.654983 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.654929 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-token\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.654983 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.654960 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-credential-socket\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.655251 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.655063 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-workload-socket\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.655251 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.655096 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-workload-certs\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.655251 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.655144 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w8kb\" (UniqueName: \"kubernetes.io/projected/332af4f4-dafa-41e4-b836-05917a3fc1b4-kube-api-access-9w8kb\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.655422 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.655398 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/332af4f4-dafa-41e4-b836-05917a3fc1b4-istiod-ca-cert\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.655422 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.655410 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-workload-certs\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.655541 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.655432 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-credential-socket\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.655541 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.655496 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-data\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.657642 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.657610 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-envoy\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.657802 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.657786 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-podinfo\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.671539 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.671510 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w8kb\" (UniqueName: \"kubernetes.io/projected/332af4f4-dafa-41e4-b836-05917a3fc1b4-kube-api-access-9w8kb\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.671711 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.671694 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-token\") pod \"e2e-ait-554b1842-openshift-default-7cc45c588-hjk52\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.780256 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.780147 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:48.915942 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.915916 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52"] Jun 15 03:44:48.917880 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:44:48.917826 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod332af4f4_dafa_41e4_b836_05917a3fc1b4.slice/crio-74ace368a84a726d34e92a1fd0da91e6373216b5c2830526082a00c7119a2de1 WatchSource:0}: Error finding container 74ace368a84a726d34e92a1fd0da91e6373216b5c2830526082a00c7119a2de1: Status 404 returned error can't find the container with id 74ace368a84a726d34e92a1fd0da91e6373216b5c2830526082a00c7119a2de1 Jun 15 03:44:48.920027 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.919993 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:44:48.920119 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.920054 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:44:48.920119 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:48.920083 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:44:49.573517 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:49.573477 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" event={"ID":"332af4f4-dafa-41e4-b836-05917a3fc1b4","Type":"ContainerStarted","Data":"ff08faa3dd9ec8bed5fdc30d3bff4b6aae3e940f3a89b35aba81b6992efc5301"} Jun 15 03:44:49.574007 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:49.573549 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" event={"ID":"332af4f4-dafa-41e4-b836-05917a3fc1b4","Type":"ContainerStarted","Data":"74ace368a84a726d34e92a1fd0da91e6373216b5c2830526082a00c7119a2de1"} Jun 15 03:44:49.594536 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:49.594484 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" podStartSLOduration=1.594469062 podStartE2EDuration="1.594469062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:44:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:44:49.593517431 +0000 UTC m=+2157.904656424" watchObservedRunningTime="2026-06-15 03:44:49.594469062 +0000 UTC m=+2157.905608051" Jun 15 03:44:49.781210 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:49.781166 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:49.786787 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:49.786759 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:50.046080 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:50.046048 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17785418-7616-4271-9e81-f7993dd0b8e2" path="/var/lib/kubelet/pods/17785418-7616-4271-9e81-f7993dd0b8e2/volumes" Jun 15 03:44:50.579923 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:50.579891 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:50.581120 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:50.581096 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:44:55.834825 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:55.834779 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52"] Jun 15 03:44:55.835377 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:44:55.834995 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" podUID="332af4f4-dafa-41e4-b836-05917a3fc1b4" containerName="istio-proxy" containerID="cri-o://ff08faa3dd9ec8bed5fdc30d3bff4b6aae3e940f3a89b35aba81b6992efc5301" gracePeriod=30 Jun 15 03:45:01.087486 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.087454 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:45:01.270630 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.270590 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-token\") pod \"332af4f4-dafa-41e4-b836-05917a3fc1b4\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " Jun 15 03:45:01.270630 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.270633 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-workload-certs\") pod \"332af4f4-dafa-41e4-b836-05917a3fc1b4\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " Jun 15 03:45:01.270957 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.270679 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-workload-socket\") pod \"332af4f4-dafa-41e4-b836-05917a3fc1b4\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " Jun 15 03:45:01.270957 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.270699 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-podinfo\") pod \"332af4f4-dafa-41e4-b836-05917a3fc1b4\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " Jun 15 03:45:01.270957 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.270719 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9w8kb\" (UniqueName: \"kubernetes.io/projected/332af4f4-dafa-41e4-b836-05917a3fc1b4-kube-api-access-9w8kb\") pod \"332af4f4-dafa-41e4-b836-05917a3fc1b4\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " Jun 15 03:45:01.270957 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.270748 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/332af4f4-dafa-41e4-b836-05917a3fc1b4-istiod-ca-cert\") pod \"332af4f4-dafa-41e4-b836-05917a3fc1b4\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " Jun 15 03:45:01.270957 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.270800 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-credential-socket\") pod \"332af4f4-dafa-41e4-b836-05917a3fc1b4\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " Jun 15 03:45:01.270957 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.270877 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-envoy\") pod \"332af4f4-dafa-41e4-b836-05917a3fc1b4\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " Jun 15 03:45:01.270957 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.270917 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-data\") pod \"332af4f4-dafa-41e4-b836-05917a3fc1b4\" (UID: \"332af4f4-dafa-41e4-b836-05917a3fc1b4\") " Jun 15 03:45:01.271307 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.271114 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "332af4f4-dafa-41e4-b836-05917a3fc1b4" (UID: "332af4f4-dafa-41e4-b836-05917a3fc1b4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:45:01.271307 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.271137 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "332af4f4-dafa-41e4-b836-05917a3fc1b4" (UID: "332af4f4-dafa-41e4-b836-05917a3fc1b4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:45:01.271307 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.271185 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/332af4f4-dafa-41e4-b836-05917a3fc1b4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "332af4f4-dafa-41e4-b836-05917a3fc1b4" (UID: "332af4f4-dafa-41e4-b836-05917a3fc1b4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:45:01.271307 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.271273 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-workload-socket\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:45:01.271502 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.271321 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/332af4f4-dafa-41e4-b836-05917a3fc1b4-istiod-ca-cert\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:45:01.271502 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.271338 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-workload-certs\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:45:01.271502 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.271349 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-data" (OuterVolumeSpecName: "istio-data") pod "332af4f4-dafa-41e4-b836-05917a3fc1b4" (UID: "332af4f4-dafa-41e4-b836-05917a3fc1b4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:45:01.271502 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.271389 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "332af4f4-dafa-41e4-b836-05917a3fc1b4" (UID: "332af4f4-dafa-41e4-b836-05917a3fc1b4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:45:01.272902 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.272882 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-token" (OuterVolumeSpecName: "istio-token") pod "332af4f4-dafa-41e4-b836-05917a3fc1b4" (UID: "332af4f4-dafa-41e4-b836-05917a3fc1b4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:45:01.273105 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.273073 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/332af4f4-dafa-41e4-b836-05917a3fc1b4-kube-api-access-9w8kb" (OuterVolumeSpecName: "kube-api-access-9w8kb") pod "332af4f4-dafa-41e4-b836-05917a3fc1b4" (UID: "332af4f4-dafa-41e4-b836-05917a3fc1b4"). InnerVolumeSpecName "kube-api-access-9w8kb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:45:01.273239 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.273218 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "332af4f4-dafa-41e4-b836-05917a3fc1b4" (UID: "332af4f4-dafa-41e4-b836-05917a3fc1b4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 15 03:45:01.273314 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.273294 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "332af4f4-dafa-41e4-b836-05917a3fc1b4" (UID: "332af4f4-dafa-41e4-b836-05917a3fc1b4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:45:01.372650 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.372615 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-credential-socket\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:45:01.372650 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.372647 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-envoy\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:45:01.372650 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.372657 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-data\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:45:01.372978 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.372665 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-token\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:45:01.372978 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.372673 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/332af4f4-dafa-41e4-b836-05917a3fc1b4-istio-podinfo\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:45:01.372978 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.372682 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9w8kb\" (UniqueName: \"kubernetes.io/projected/332af4f4-dafa-41e4-b836-05917a3fc1b4-kube-api-access-9w8kb\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:45:01.635635 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.635601 2562 generic.go:356] "Generic (PLEG): container finished" podID="332af4f4-dafa-41e4-b836-05917a3fc1b4" containerID="ff08faa3dd9ec8bed5fdc30d3bff4b6aae3e940f3a89b35aba81b6992efc5301" exitCode=0 Jun 15 03:45:01.635809 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.635650 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" Jun 15 03:45:01.635809 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.635679 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" event={"ID":"332af4f4-dafa-41e4-b836-05917a3fc1b4","Type":"ContainerDied","Data":"ff08faa3dd9ec8bed5fdc30d3bff4b6aae3e940f3a89b35aba81b6992efc5301"} Jun 15 03:45:01.635809 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.635734 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52" event={"ID":"332af4f4-dafa-41e4-b836-05917a3fc1b4","Type":"ContainerDied","Data":"74ace368a84a726d34e92a1fd0da91e6373216b5c2830526082a00c7119a2de1"} Jun 15 03:45:01.635809 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.635757 2562 scope.go:117] "RemoveContainer" containerID="ff08faa3dd9ec8bed5fdc30d3bff4b6aae3e940f3a89b35aba81b6992efc5301" Jun 15 03:45:01.644713 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.644695 2562 scope.go:117] "RemoveContainer" containerID="ff08faa3dd9ec8bed5fdc30d3bff4b6aae3e940f3a89b35aba81b6992efc5301" Jun 15 03:45:01.644994 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:45:01.644971 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff08faa3dd9ec8bed5fdc30d3bff4b6aae3e940f3a89b35aba81b6992efc5301\": container with ID starting with ff08faa3dd9ec8bed5fdc30d3bff4b6aae3e940f3a89b35aba81b6992efc5301 not found: ID does not exist" containerID="ff08faa3dd9ec8bed5fdc30d3bff4b6aae3e940f3a89b35aba81b6992efc5301" Jun 15 03:45:01.645068 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.645004 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff08faa3dd9ec8bed5fdc30d3bff4b6aae3e940f3a89b35aba81b6992efc5301"} err="failed to get container status \"ff08faa3dd9ec8bed5fdc30d3bff4b6aae3e940f3a89b35aba81b6992efc5301\": rpc error: code = NotFound desc = could not find container \"ff08faa3dd9ec8bed5fdc30d3bff4b6aae3e940f3a89b35aba81b6992efc5301\": container with ID starting with ff08faa3dd9ec8bed5fdc30d3bff4b6aae3e940f3a89b35aba81b6992efc5301 not found: ID does not exist" Jun 15 03:45:01.656606 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.656582 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52"] Jun 15 03:45:01.661252 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.661229 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-554b1842-openshift-default-7cc45c588-hjk52"] Jun 15 03:45:01.787913 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.787877 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq"] Jun 15 03:45:01.788633 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.788605 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="332af4f4-dafa-41e4-b836-05917a3fc1b4" containerName="istio-proxy" Jun 15 03:45:01.788633 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.788635 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="332af4f4-dafa-41e4-b836-05917a3fc1b4" containerName="istio-proxy" Jun 15 03:45:01.788785 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.788742 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="332af4f4-dafa-41e4-b836-05917a3fc1b4" containerName="istio-proxy" Jun 15 03:45:01.794054 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.794030 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:01.796475 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.796452 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-invalid-7a50aa40-openshift-default-dockercfg-lb8ps\"" Jun 15 03:45:01.804348 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.804321 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq"] Jun 15 03:45:01.979189 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.979091 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-podinfo\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:01.979189 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.979128 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-token\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:01.979189 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.979157 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-credential-socket\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:01.979189 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.979186 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-envoy\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:01.979531 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.979243 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-workload-certs\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:01.979531 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.979298 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e00288cb-f92c-491d-825f-8cae6dff8dbe-istiod-ca-cert\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:01.979531 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.979359 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxtm4\" (UniqueName: \"kubernetes.io/projected/e00288cb-f92c-491d-825f-8cae6dff8dbe-kube-api-access-rxtm4\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:01.979531 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.979435 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-workload-socket\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:01.979531 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:01.979468 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-data\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:02.045559 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.045526 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="332af4f4-dafa-41e4-b836-05917a3fc1b4" path="/var/lib/kubelet/pods/332af4f4-dafa-41e4-b836-05917a3fc1b4/volumes" Jun 15 03:45:02.079941 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.079902 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e00288cb-f92c-491d-825f-8cae6dff8dbe-istiod-ca-cert\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:02.080085 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.079950 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxtm4\" (UniqueName: \"kubernetes.io/projected/e00288cb-f92c-491d-825f-8cae6dff8dbe-kube-api-access-rxtm4\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:02.080085 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.080009 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-workload-socket\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:02.080085 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.080028 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-data\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:02.080085 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.080057 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-podinfo\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:02.080085 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.080074 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-token\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:02.080351 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.080108 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-credential-socket\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:02.080351 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.080132 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-envoy\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:02.080351 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.080164 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-workload-certs\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:02.080518 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.080460 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-data\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:02.080518 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.080503 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-workload-socket\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:02.080638 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.080615 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-credential-socket\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:02.080802 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.080777 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-workload-certs\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:02.081053 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.081031 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e00288cb-f92c-491d-825f-8cae6dff8dbe-istiod-ca-cert\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:02.082504 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.082483 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-envoy\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:02.082757 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.082736 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-podinfo\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:02.087461 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.087433 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-token\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:02.087552 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.087521 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxtm4\" (UniqueName: \"kubernetes.io/projected/e00288cb-f92c-491d-825f-8cae6dff8dbe-kube-api-access-rxtm4\") pod \"e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:02.107065 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.107043 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:02.241609 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.241526 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq"] Jun 15 03:45:02.245150 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:45:02.245125 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode00288cb_f92c_491d_825f_8cae6dff8dbe.slice/crio-c21056a4657e0b2456aa81a9e75f078c976e10152f59c1dbb226048b99da6bb0 WatchSource:0}: Error finding container c21056a4657e0b2456aa81a9e75f078c976e10152f59c1dbb226048b99da6bb0: Status 404 returned error can't find the container with id c21056a4657e0b2456aa81a9e75f078c976e10152f59c1dbb226048b99da6bb0 Jun 15 03:45:02.247335 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.247304 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:45:02.247405 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.247374 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:45:02.247445 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.247403 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:45:02.349091 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.349053 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq"] Jun 15 03:45:02.641776 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.641739 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" event={"ID":"e00288cb-f92c-491d-825f-8cae6dff8dbe","Type":"ContainerStarted","Data":"b91b895c3d09d112cdc86c5735026c9528ca5419ab8abf87eaf956ac0a99bf49"} Jun 15 03:45:02.641977 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.641793 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" event={"ID":"e00288cb-f92c-491d-825f-8cae6dff8dbe","Type":"ContainerStarted","Data":"c21056a4657e0b2456aa81a9e75f078c976e10152f59c1dbb226048b99da6bb0"} Jun 15 03:45:02.642029 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.641994 2562 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" secret="" err="secret \"e2e-invalid-7a50aa40-openshift-default-dockercfg-lb8ps\" not found" Jun 15 03:45:02.659356 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:02.659300 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" podStartSLOduration=1.659281556 podStartE2EDuration="1.659281556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:45:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:45:02.658260547 +0000 UTC m=+2170.969399536" watchObservedRunningTime="2026-06-15 03:45:02.659281556 +0000 UTC m=+2170.970420550" Jun 15 03:45:03.107216 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:03.107181 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:03.108511 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:03.108485 2562 patch_prober.go:28] interesting pod/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jun 15 03:45:03.108644 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:03.108549 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" podUID="e00288cb-f92c-491d-825f-8cae6dff8dbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jun 15 03:45:03.647251 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:03.647214 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" podUID="e00288cb-f92c-491d-825f-8cae6dff8dbe" containerName="istio-proxy" containerID="cri-o://b91b895c3d09d112cdc86c5735026c9528ca5419ab8abf87eaf956ac0a99bf49" gracePeriod=30 Jun 15 03:45:08.918753 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.918729 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:08.939371 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.939342 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e00288cb-f92c-491d-825f-8cae6dff8dbe-istiod-ca-cert\") pod \"e00288cb-f92c-491d-825f-8cae6dff8dbe\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " Jun 15 03:45:08.939502 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.939385 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-podinfo\") pod \"e00288cb-f92c-491d-825f-8cae6dff8dbe\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " Jun 15 03:45:08.939502 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.939409 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-token\") pod \"e00288cb-f92c-491d-825f-8cae6dff8dbe\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " Jun 15 03:45:08.939593 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.939579 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-envoy\") pod \"e00288cb-f92c-491d-825f-8cae6dff8dbe\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " Jun 15 03:45:08.939635 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.939612 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxtm4\" (UniqueName: \"kubernetes.io/projected/e00288cb-f92c-491d-825f-8cae6dff8dbe-kube-api-access-rxtm4\") pod \"e00288cb-f92c-491d-825f-8cae6dff8dbe\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " Jun 15 03:45:08.939685 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.939646 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-data\") pod \"e00288cb-f92c-491d-825f-8cae6dff8dbe\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " Jun 15 03:45:08.939750 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.939719 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-workload-socket\") pod \"e00288cb-f92c-491d-825f-8cae6dff8dbe\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " Jun 15 03:45:08.940010 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.939983 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-workload-certs\") pod \"e00288cb-f92c-491d-825f-8cae6dff8dbe\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " Jun 15 03:45:08.940121 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.940025 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-credential-socket\") pod \"e00288cb-f92c-491d-825f-8cae6dff8dbe\" (UID: \"e00288cb-f92c-491d-825f-8cae6dff8dbe\") " Jun 15 03:45:08.940121 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.939743 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e00288cb-f92c-491d-825f-8cae6dff8dbe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e00288cb-f92c-491d-825f-8cae6dff8dbe" (UID: "e00288cb-f92c-491d-825f-8cae6dff8dbe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:45:08.940121 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.939964 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e00288cb-f92c-491d-825f-8cae6dff8dbe" (UID: "e00288cb-f92c-491d-825f-8cae6dff8dbe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:45:08.940121 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.939984 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-data" (OuterVolumeSpecName: "istio-data") pod "e00288cb-f92c-491d-825f-8cae6dff8dbe" (UID: "e00288cb-f92c-491d-825f-8cae6dff8dbe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:45:08.941780 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.940292 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e00288cb-f92c-491d-825f-8cae6dff8dbe" (UID: "e00288cb-f92c-491d-825f-8cae6dff8dbe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:45:08.941780 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.940491 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-data\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:45:08.941780 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.940513 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-workload-socket\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:45:08.941780 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.940527 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-workload-certs\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:45:08.941780 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.940540 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e00288cb-f92c-491d-825f-8cae6dff8dbe-istiod-ca-cert\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:45:08.941780 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.941526 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e00288cb-f92c-491d-825f-8cae6dff8dbe" (UID: "e00288cb-f92c-491d-825f-8cae6dff8dbe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:45:08.942683 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.942281 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e00288cb-f92c-491d-825f-8cae6dff8dbe" (UID: "e00288cb-f92c-491d-825f-8cae6dff8dbe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 15 03:45:08.942683 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.942396 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e00288cb-f92c-491d-825f-8cae6dff8dbe-kube-api-access-rxtm4" (OuterVolumeSpecName: "kube-api-access-rxtm4") pod "e00288cb-f92c-491d-825f-8cae6dff8dbe" (UID: "e00288cb-f92c-491d-825f-8cae6dff8dbe"). InnerVolumeSpecName "kube-api-access-rxtm4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:45:08.942883 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.942802 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-token" (OuterVolumeSpecName: "istio-token") pod "e00288cb-f92c-491d-825f-8cae6dff8dbe" (UID: "e00288cb-f92c-491d-825f-8cae6dff8dbe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:45:08.942883 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:08.942818 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e00288cb-f92c-491d-825f-8cae6dff8dbe" (UID: "e00288cb-f92c-491d-825f-8cae6dff8dbe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:45:09.040927 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:09.040894 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-podinfo\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:45:09.040927 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:09.040919 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-token\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:45:09.040927 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:09.040928 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-istio-envoy\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:45:09.041178 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:09.040938 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxtm4\" (UniqueName: \"kubernetes.io/projected/e00288cb-f92c-491d-825f-8cae6dff8dbe-kube-api-access-rxtm4\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:45:09.041178 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:09.040948 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e00288cb-f92c-491d-825f-8cae6dff8dbe-credential-socket\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:45:09.676019 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:09.675983 2562 generic.go:356] "Generic (PLEG): container finished" podID="e00288cb-f92c-491d-825f-8cae6dff8dbe" containerID="b91b895c3d09d112cdc86c5735026c9528ca5419ab8abf87eaf956ac0a99bf49" exitCode=0 Jun 15 03:45:09.676202 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:09.676040 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" Jun 15 03:45:09.676202 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:09.676058 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" event={"ID":"e00288cb-f92c-491d-825f-8cae6dff8dbe","Type":"ContainerDied","Data":"b91b895c3d09d112cdc86c5735026c9528ca5419ab8abf87eaf956ac0a99bf49"} Jun 15 03:45:09.676202 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:09.676124 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq" event={"ID":"e00288cb-f92c-491d-825f-8cae6dff8dbe","Type":"ContainerDied","Data":"c21056a4657e0b2456aa81a9e75f078c976e10152f59c1dbb226048b99da6bb0"} Jun 15 03:45:09.676202 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:09.676147 2562 scope.go:117] "RemoveContainer" containerID="b91b895c3d09d112cdc86c5735026c9528ca5419ab8abf87eaf956ac0a99bf49" Jun 15 03:45:09.687790 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:09.687771 2562 scope.go:117] "RemoveContainer" containerID="b91b895c3d09d112cdc86c5735026c9528ca5419ab8abf87eaf956ac0a99bf49" Jun 15 03:45:09.688068 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:45:09.688045 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b91b895c3d09d112cdc86c5735026c9528ca5419ab8abf87eaf956ac0a99bf49\": container with ID starting with b91b895c3d09d112cdc86c5735026c9528ca5419ab8abf87eaf956ac0a99bf49 not found: ID does not exist" containerID="b91b895c3d09d112cdc86c5735026c9528ca5419ab8abf87eaf956ac0a99bf49" Jun 15 03:45:09.688158 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:09.688087 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b91b895c3d09d112cdc86c5735026c9528ca5419ab8abf87eaf956ac0a99bf49"} err="failed to get container status \"b91b895c3d09d112cdc86c5735026c9528ca5419ab8abf87eaf956ac0a99bf49\": rpc error: code = NotFound desc = could not find container \"b91b895c3d09d112cdc86c5735026c9528ca5419ab8abf87eaf956ac0a99bf49\": container with ID starting with b91b895c3d09d112cdc86c5735026c9528ca5419ab8abf87eaf956ac0a99bf49 not found: ID does not exist" Jun 15 03:45:09.698210 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:09.698180 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq"] Jun 15 03:45:09.704084 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:09.704058 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-invalid-7a50aa40-openshift-default-f459897bb-k98fq"] Jun 15 03:45:10.045181 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:45:10.045096 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e00288cb-f92c-491d-825f-8cae6dff8dbe" path="/var/lib/kubelet/pods/e00288cb-f92c-491d-825f-8cae6dff8dbe/volumes" Jun 15 03:46:31.828786 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:31.828746 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:46:35.237639 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:35.237601 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:46:38.823074 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:38.823036 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:46:43.625899 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.625864 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj"] Jun 15 03:46:43.628173 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.626218 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e00288cb-f92c-491d-825f-8cae6dff8dbe" containerName="istio-proxy" Jun 15 03:46:43.628173 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.626230 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e00288cb-f92c-491d-825f-8cae6dff8dbe" containerName="istio-proxy" Jun 15 03:46:43.628173 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.626345 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e00288cb-f92c-491d-825f-8cae6dff8dbe" containerName="istio-proxy" Jun 15 03:46:43.629489 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.629464 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.632223 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.632200 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-496f8e4e-openshift-default-dockercfg-86vph\"" Jun 15 03:46:43.647218 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.647196 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj"] Jun 15 03:46:43.672552 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.672523 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-workload-certs\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.672674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.672594 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-data\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.672674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.672628 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-token\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.672674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.672652 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-podinfo\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.672799 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.672768 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-workload-socket\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.672859 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.672800 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-envoy\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.672902 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.672891 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2gvc\" (UniqueName: \"kubernetes.io/projected/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-kube-api-access-w2gvc\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.672951 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.672942 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-credential-socket\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.673002 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.672964 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istiod-ca-cert\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.773503 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.773467 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-workload-socket\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.773503 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.773507 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-envoy\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.773755 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.773536 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2gvc\" (UniqueName: \"kubernetes.io/projected/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-kube-api-access-w2gvc\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.773755 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.773576 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-credential-socket\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.773755 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.773600 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istiod-ca-cert\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.773755 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.773642 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-workload-certs\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.773755 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.773684 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-data\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.773755 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.773713 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-token\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.773755 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.773743 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-podinfo\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.774224 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.773966 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-workload-socket\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.774224 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.774140 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-credential-socket\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.774442 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.774380 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istiod-ca-cert\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.774442 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.774413 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-data\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.774605 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.774563 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-workload-certs\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.776069 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.776039 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-envoy\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.776304 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.776284 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-podinfo\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.792608 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.792582 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-token\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.792730 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.792588 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2gvc\" (UniqueName: \"kubernetes.io/projected/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-kube-api-access-w2gvc\") pod \"e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:43.940034 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:43.939952 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:44.076828 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:44.076799 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj"] Jun 15 03:46:44.079296 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:46:44.079271 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ffe054_8fe1_4af3_b2ee_6c0baca1de62.slice/crio-e21a58f17a9e42a0bd2a445775444ffac47454d5a4873e0961d976121ac8e259 WatchSource:0}: Error finding container e21a58f17a9e42a0bd2a445775444ffac47454d5a4873e0961d976121ac8e259: Status 404 returned error can't find the container with id e21a58f17a9e42a0bd2a445775444ffac47454d5a4873e0961d976121ac8e259 Jun 15 03:46:44.081349 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:44.081319 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:46:44.081409 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:44.081379 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:46:44.081447 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:44.081410 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:46:44.107070 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:44.107041 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" event={"ID":"07ffe054-8fe1-4af3-b2ee-6c0baca1de62","Type":"ContainerStarted","Data":"e21a58f17a9e42a0bd2a445775444ffac47454d5a4873e0961d976121ac8e259"} Jun 15 03:46:45.113408 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:45.113366 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" event={"ID":"07ffe054-8fe1-4af3-b2ee-6c0baca1de62","Type":"ContainerStarted","Data":"acd5f4f4cb1c636a293974f304d21961aafd85c54044620832af0bbd40581201"} Jun 15 03:46:45.134399 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:45.134350 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" podStartSLOduration=2.134336504 podStartE2EDuration="2.134336504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:46:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:46:45.132766145 +0000 UTC m=+2273.443905127" watchObservedRunningTime="2026-06-15 03:46:45.134336504 +0000 UTC m=+2273.445475494" Jun 15 03:46:45.940474 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:45.940440 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:45.945216 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:45.945189 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:46.118964 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:46.118936 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:46:46.119944 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:46:46.119924 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:47:02.483409 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:02.483373 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj"] Jun 15 03:47:02.483925 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:02.483607 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" podUID="07ffe054-8fe1-4af3-b2ee-6c0baca1de62" containerName="istio-proxy" containerID="cri-o://acd5f4f4cb1c636a293974f304d21961aafd85c54044620832af0bbd40581201" gracePeriod=30 Jun 15 03:47:07.728564 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.728536 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:47:07.789476 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.789452 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-data\") pod \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " Jun 15 03:47:07.789621 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.789491 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-workload-socket\") pod \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " Jun 15 03:47:07.789621 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.789542 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2gvc\" (UniqueName: \"kubernetes.io/projected/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-kube-api-access-w2gvc\") pod \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " Jun 15 03:47:07.789621 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.789613 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-credential-socket\") pod \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " Jun 15 03:47:07.789793 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.789633 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-workload-certs\") pod \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " Jun 15 03:47:07.789793 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.789672 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-token\") pod \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " Jun 15 03:47:07.789793 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.789701 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-podinfo\") pod \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " Jun 15 03:47:07.789793 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.789754 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istiod-ca-cert\") pod \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " Jun 15 03:47:07.789793 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.789763 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "07ffe054-8fe1-4af3-b2ee-6c0baca1de62" (UID: "07ffe054-8fe1-4af3-b2ee-6c0baca1de62"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:47:07.789793 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.789786 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-envoy\") pod \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\" (UID: \"07ffe054-8fe1-4af3-b2ee-6c0baca1de62\") " Jun 15 03:47:07.790175 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.789787 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-data" (OuterVolumeSpecName: "istio-data") pod "07ffe054-8fe1-4af3-b2ee-6c0baca1de62" (UID: "07ffe054-8fe1-4af3-b2ee-6c0baca1de62"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:47:07.790175 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.789898 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "07ffe054-8fe1-4af3-b2ee-6c0baca1de62" (UID: "07ffe054-8fe1-4af3-b2ee-6c0baca1de62"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:47:07.790175 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.790056 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "07ffe054-8fe1-4af3-b2ee-6c0baca1de62" (UID: "07ffe054-8fe1-4af3-b2ee-6c0baca1de62"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:47:07.790353 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.790216 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-data\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:47:07.790353 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.790221 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "07ffe054-8fe1-4af3-b2ee-6c0baca1de62" (UID: "07ffe054-8fe1-4af3-b2ee-6c0baca1de62"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:47:07.790353 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.790235 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-workload-socket\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:47:07.790353 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.790249 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-credential-socket\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:47:07.790353 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.790263 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-workload-certs\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:47:07.792156 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.792121 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-token" (OuterVolumeSpecName: "istio-token") pod "07ffe054-8fe1-4af3-b2ee-6c0baca1de62" (UID: "07ffe054-8fe1-4af3-b2ee-6c0baca1de62"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:47:07.792258 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.792149 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "07ffe054-8fe1-4af3-b2ee-6c0baca1de62" (UID: "07ffe054-8fe1-4af3-b2ee-6c0baca1de62"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 15 03:47:07.792585 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.792558 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "07ffe054-8fe1-4af3-b2ee-6c0baca1de62" (UID: "07ffe054-8fe1-4af3-b2ee-6c0baca1de62"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:47:07.792585 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.792560 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-kube-api-access-w2gvc" (OuterVolumeSpecName: "kube-api-access-w2gvc") pod "07ffe054-8fe1-4af3-b2ee-6c0baca1de62" (UID: "07ffe054-8fe1-4af3-b2ee-6c0baca1de62"). InnerVolumeSpecName "kube-api-access-w2gvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:47:07.891013 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.890983 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-token\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:47:07.891013 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.891009 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-podinfo\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:47:07.891013 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.891018 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istiod-ca-cert\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:47:07.891218 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.891026 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-istio-envoy\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:47:07.891218 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:07.891035 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2gvc\" (UniqueName: \"kubernetes.io/projected/07ffe054-8fe1-4af3-b2ee-6c0baca1de62-kube-api-access-w2gvc\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:47:08.223796 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:08.223760 2562 generic.go:356] "Generic (PLEG): container finished" podID="07ffe054-8fe1-4af3-b2ee-6c0baca1de62" containerID="acd5f4f4cb1c636a293974f304d21961aafd85c54044620832af0bbd40581201" exitCode=0 Jun 15 03:47:08.223988 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:08.223803 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" Jun 15 03:47:08.223988 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:08.223863 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" event={"ID":"07ffe054-8fe1-4af3-b2ee-6c0baca1de62","Type":"ContainerDied","Data":"acd5f4f4cb1c636a293974f304d21961aafd85c54044620832af0bbd40581201"} Jun 15 03:47:08.223988 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:08.223921 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj" event={"ID":"07ffe054-8fe1-4af3-b2ee-6c0baca1de62","Type":"ContainerDied","Data":"e21a58f17a9e42a0bd2a445775444ffac47454d5a4873e0961d976121ac8e259"} Jun 15 03:47:08.223988 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:08.223949 2562 scope.go:117] "RemoveContainer" containerID="acd5f4f4cb1c636a293974f304d21961aafd85c54044620832af0bbd40581201" Jun 15 03:47:08.232860 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:08.232805 2562 scope.go:117] "RemoveContainer" containerID="acd5f4f4cb1c636a293974f304d21961aafd85c54044620832af0bbd40581201" Jun 15 03:47:08.233116 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:47:08.233098 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acd5f4f4cb1c636a293974f304d21961aafd85c54044620832af0bbd40581201\": container with ID starting with acd5f4f4cb1c636a293974f304d21961aafd85c54044620832af0bbd40581201 not found: ID does not exist" containerID="acd5f4f4cb1c636a293974f304d21961aafd85c54044620832af0bbd40581201" Jun 15 03:47:08.233174 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:08.233126 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acd5f4f4cb1c636a293974f304d21961aafd85c54044620832af0bbd40581201"} err="failed to get container status \"acd5f4f4cb1c636a293974f304d21961aafd85c54044620832af0bbd40581201\": rpc error: code = NotFound desc = could not find container \"acd5f4f4cb1c636a293974f304d21961aafd85c54044620832af0bbd40581201\": container with ID starting with acd5f4f4cb1c636a293974f304d21961aafd85c54044620832af0bbd40581201 not found: ID does not exist" Jun 15 03:47:08.239785 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:08.239762 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj"] Jun 15 03:47:08.243624 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:08.243604 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-496f8e4e-openshift-default-75d8577c5d-s6xpj"] Jun 15 03:47:10.045974 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:10.045939 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07ffe054-8fe1-4af3-b2ee-6c0baca1de62" path="/var/lib/kubelet/pods/07ffe054-8fe1-4af3-b2ee-6c0baca1de62/volumes" Jun 15 03:47:24.950978 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:24.950947 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww"] Jun 15 03:47:24.951339 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:24.951316 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07ffe054-8fe1-4af3-b2ee-6c0baca1de62" containerName="istio-proxy" Jun 15 03:47:24.951339 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:24.951330 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="07ffe054-8fe1-4af3-b2ee-6c0baca1de62" containerName="istio-proxy" Jun 15 03:47:24.951416 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:24.951402 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="07ffe054-8fe1-4af3-b2ee-6c0baca1de62" containerName="istio-proxy" Jun 15 03:47:24.954387 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:24.954365 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:24.956424 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:24.956403 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-89177eef-openshift-default-dockercfg-dzl6d\"" Jun 15 03:47:24.969199 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:24.969173 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww"] Jun 15 03:47:25.135208 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.135169 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8sfc\" (UniqueName: \"kubernetes.io/projected/8f820240-f63b-40ff-b046-dea2937568aa-kube-api-access-d8sfc\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.135381 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.135309 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-workload-socket\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.135381 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.135355 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-istio-envoy\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.135381 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.135371 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-istio-data\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.135541 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.135394 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f820240-f63b-40ff-b046-dea2937568aa-istiod-ca-cert\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.135620 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.135605 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-credential-socket\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.135688 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.135635 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f820240-f63b-40ff-b046-dea2937568aa-istio-token\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.135688 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.135669 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f820240-f63b-40ff-b046-dea2937568aa-istio-podinfo\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.135796 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.135705 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-workload-certs\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.236647 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.236564 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8sfc\" (UniqueName: \"kubernetes.io/projected/8f820240-f63b-40ff-b046-dea2937568aa-kube-api-access-d8sfc\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.236790 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.236662 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-workload-socket\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.236790 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.236696 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-istio-envoy\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.236790 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.236718 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-istio-data\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.236790 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.236743 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f820240-f63b-40ff-b046-dea2937568aa-istiod-ca-cert\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.236790 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.236785 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-credential-socket\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.237070 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.236818 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f820240-f63b-40ff-b046-dea2937568aa-istio-token\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.237070 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.236879 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f820240-f63b-40ff-b046-dea2937568aa-istio-podinfo\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.237070 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.236905 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-workload-certs\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.237227 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.237141 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-workload-socket\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.237310 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.237285 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-istio-data\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.237368 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.237350 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-credential-socket\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.237482 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.237448 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-workload-certs\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.237742 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.237720 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f820240-f63b-40ff-b046-dea2937568aa-istiod-ca-cert\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.239252 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.239232 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-istio-envoy\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.239562 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.239542 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f820240-f63b-40ff-b046-dea2937568aa-istio-podinfo\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.245344 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.245323 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f820240-f63b-40ff-b046-dea2937568aa-istio-token\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.245539 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.245520 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8sfc\" (UniqueName: \"kubernetes.io/projected/8f820240-f63b-40ff-b046-dea2937568aa-kube-api-access-d8sfc\") pod \"e2e-mt-89177eef-openshift-default-7dcb564874-p9mww\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.265320 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.265293 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:25.398508 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.398480 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww"] Jun 15 03:47:25.400579 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:47:25.400553 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f820240_f63b_40ff_b046_dea2937568aa.slice/crio-d5a21d7584d41e419fd476ded12c9bf06d92233a2d31b1b807b25deba6c3d3a9 WatchSource:0}: Error finding container d5a21d7584d41e419fd476ded12c9bf06d92233a2d31b1b807b25deba6c3d3a9: Status 404 returned error can't find the container with id d5a21d7584d41e419fd476ded12c9bf06d92233a2d31b1b807b25deba6c3d3a9 Jun 15 03:47:25.402644 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.402615 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:47:25.402741 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.402678 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:47:25.402741 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:25.402723 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 15 03:47:26.306640 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:26.306603 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" event={"ID":"8f820240-f63b-40ff-b046-dea2937568aa","Type":"ContainerStarted","Data":"852776528e20d17770a15c6aa6a98f035f7230b3cba5061c8b352238d50872cd"} Jun 15 03:47:26.307032 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:26.306669 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" event={"ID":"8f820240-f63b-40ff-b046-dea2937568aa","Type":"ContainerStarted","Data":"d5a21d7584d41e419fd476ded12c9bf06d92233a2d31b1b807b25deba6c3d3a9"} Jun 15 03:47:26.326482 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:26.326437 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" podStartSLOduration=2.326423336 podStartE2EDuration="2.326423336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:47:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:47:26.324981274 +0000 UTC m=+2314.636120266" watchObservedRunningTime="2026-06-15 03:47:26.326423336 +0000 UTC m=+2314.637562325" Jun 15 03:47:27.266439 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:27.266407 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:27.270971 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:27.270950 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:27.310762 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:27.310740 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:47:27.311596 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:47:27.311581 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:49:40.913848 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:40.913809 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww"] Jun 15 03:49:40.914429 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:40.914086 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" podUID="8f820240-f63b-40ff-b046-dea2937568aa" containerName="istio-proxy" containerID="cri-o://852776528e20d17770a15c6aa6a98f035f7230b3cba5061c8b352238d50872cd" gracePeriod=30 Jun 15 03:49:42.313526 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:42.313484 2562 patch_prober.go:28] interesting pod/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jun 15 03:49:42.313915 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:42.313553 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" podUID="8f820240-f63b-40ff-b046-dea2937568aa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 15 03:49:46.037555 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.037514 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:49:46.164372 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.164347 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:49:46.298095 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.298038 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8sfc\" (UniqueName: \"kubernetes.io/projected/8f820240-f63b-40ff-b046-dea2937568aa-kube-api-access-d8sfc\") pod \"8f820240-f63b-40ff-b046-dea2937568aa\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " Jun 15 03:49:46.298270 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.298101 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f820240-f63b-40ff-b046-dea2937568aa-istiod-ca-cert\") pod \"8f820240-f63b-40ff-b046-dea2937568aa\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " Jun 15 03:49:46.298270 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.298160 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-workload-socket\") pod \"8f820240-f63b-40ff-b046-dea2937568aa\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " Jun 15 03:49:46.298270 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.298177 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f820240-f63b-40ff-b046-dea2937568aa-istio-token\") pod \"8f820240-f63b-40ff-b046-dea2937568aa\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " Jun 15 03:49:46.298270 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.298197 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-credential-socket\") pod \"8f820240-f63b-40ff-b046-dea2937568aa\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " Jun 15 03:49:46.298524 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.298281 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f820240-f63b-40ff-b046-dea2937568aa-istio-podinfo\") pod \"8f820240-f63b-40ff-b046-dea2937568aa\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " Jun 15 03:49:46.298524 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.298314 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-workload-certs\") pod \"8f820240-f63b-40ff-b046-dea2937568aa\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " Jun 15 03:49:46.298524 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.298349 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-istio-envoy\") pod \"8f820240-f63b-40ff-b046-dea2937568aa\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " Jun 15 03:49:46.298524 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.298380 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-istio-data\") pod \"8f820240-f63b-40ff-b046-dea2937568aa\" (UID: \"8f820240-f63b-40ff-b046-dea2937568aa\") " Jun 15 03:49:46.298524 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.298407 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8f820240-f63b-40ff-b046-dea2937568aa" (UID: "8f820240-f63b-40ff-b046-dea2937568aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:49:46.298806 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.298595 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8f820240-f63b-40ff-b046-dea2937568aa" (UID: "8f820240-f63b-40ff-b046-dea2937568aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:49:46.298806 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.298614 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8f820240-f63b-40ff-b046-dea2937568aa" (UID: "8f820240-f63b-40ff-b046-dea2937568aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:49:46.298806 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.298708 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f820240-f63b-40ff-b046-dea2937568aa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8f820240-f63b-40ff-b046-dea2937568aa" (UID: "8f820240-f63b-40ff-b046-dea2937568aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 15 03:49:46.299001 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.298752 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-istio-data" (OuterVolumeSpecName: "istio-data") pod "8f820240-f63b-40ff-b046-dea2937568aa" (UID: "8f820240-f63b-40ff-b046-dea2937568aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:49:46.299001 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.298955 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f820240-f63b-40ff-b046-dea2937568aa-istiod-ca-cert\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:49:46.299001 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.298976 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-workload-socket\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:49:46.299001 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.298990 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-credential-socket\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:49:46.299216 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.299021 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-workload-certs\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:49:46.299216 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.299037 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-istio-data\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:49:46.300557 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.300523 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8f820240-f63b-40ff-b046-dea2937568aa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8f820240-f63b-40ff-b046-dea2937568aa" (UID: "8f820240-f63b-40ff-b046-dea2937568aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 15 03:49:46.300557 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.300543 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f820240-f63b-40ff-b046-dea2937568aa-istio-token" (OuterVolumeSpecName: "istio-token") pod "8f820240-f63b-40ff-b046-dea2937568aa" (UID: "8f820240-f63b-40ff-b046-dea2937568aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:49:46.300737 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.300588 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f820240-f63b-40ff-b046-dea2937568aa-kube-api-access-d8sfc" (OuterVolumeSpecName: "kube-api-access-d8sfc") pod "8f820240-f63b-40ff-b046-dea2937568aa" (UID: "8f820240-f63b-40ff-b046-dea2937568aa"). InnerVolumeSpecName "kube-api-access-d8sfc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 15 03:49:46.300737 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.300637 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8f820240-f63b-40ff-b046-dea2937568aa" (UID: "8f820240-f63b-40ff-b046-dea2937568aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 15 03:49:46.399881 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.399851 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f820240-f63b-40ff-b046-dea2937568aa-istio-token\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:49:46.399881 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.399878 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f820240-f63b-40ff-b046-dea2937568aa-istio-podinfo\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:49:46.400048 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.399887 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f820240-f63b-40ff-b046-dea2937568aa-istio-envoy\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:49:46.400048 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.399897 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8sfc\" (UniqueName: \"kubernetes.io/projected/8f820240-f63b-40ff-b046-dea2937568aa-kube-api-access-d8sfc\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jun 15 03:49:46.977916 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.977882 2562 generic.go:356] "Generic (PLEG): container finished" podID="8f820240-f63b-40ff-b046-dea2937568aa" containerID="852776528e20d17770a15c6aa6a98f035f7230b3cba5061c8b352238d50872cd" exitCode=0 Jun 15 03:49:46.978103 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.977936 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" Jun 15 03:49:46.978103 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.977947 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" event={"ID":"8f820240-f63b-40ff-b046-dea2937568aa","Type":"ContainerDied","Data":"852776528e20d17770a15c6aa6a98f035f7230b3cba5061c8b352238d50872cd"} Jun 15 03:49:46.978103 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.977986 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww" event={"ID":"8f820240-f63b-40ff-b046-dea2937568aa","Type":"ContainerDied","Data":"d5a21d7584d41e419fd476ded12c9bf06d92233a2d31b1b807b25deba6c3d3a9"} Jun 15 03:49:46.978103 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.978007 2562 scope.go:117] "RemoveContainer" containerID="852776528e20d17770a15c6aa6a98f035f7230b3cba5061c8b352238d50872cd" Jun 15 03:49:46.987143 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.987118 2562 scope.go:117] "RemoveContainer" containerID="852776528e20d17770a15c6aa6a98f035f7230b3cba5061c8b352238d50872cd" Jun 15 03:49:46.987378 ip-10-0-143-185 kubenswrapper[2562]: E0615 03:49:46.987359 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"852776528e20d17770a15c6aa6a98f035f7230b3cba5061c8b352238d50872cd\": container with ID starting with 852776528e20d17770a15c6aa6a98f035f7230b3cba5061c8b352238d50872cd not found: ID does not exist" containerID="852776528e20d17770a15c6aa6a98f035f7230b3cba5061c8b352238d50872cd" Jun 15 03:49:46.987439 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.987388 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"852776528e20d17770a15c6aa6a98f035f7230b3cba5061c8b352238d50872cd"} err="failed to get container status \"852776528e20d17770a15c6aa6a98f035f7230b3cba5061c8b352238d50872cd\": rpc error: code = NotFound desc = could not find container \"852776528e20d17770a15c6aa6a98f035f7230b3cba5061c8b352238d50872cd\": container with ID starting with 852776528e20d17770a15c6aa6a98f035f7230b3cba5061c8b352238d50872cd not found: ID does not exist" Jun 15 03:49:46.998929 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:46.998905 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww"] Jun 15 03:49:47.000505 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:47.000485 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-89177eef-openshift-default-7dcb564874-p9mww"] Jun 15 03:49:48.044715 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:48.044677 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f820240-f63b-40ff-b046-dea2937568aa" path="/var/lib/kubelet/pods/8f820240-f63b-40ff-b046-dea2937568aa/volumes" Jun 15 03:49:52.125313 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:49:52.125279 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lcq9d"] Jun 15 03:54:22.616851 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:22.616816 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-694cb6497d-45zn5_277599cb-8940-479c-a249-5244193660d4/nginx/0.log" Jun 15 03:54:23.820217 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:23.820187 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5799f64c8f-fk5zd_df113ca2-256b-46fb-aec0-8693e938e085/copy-manifests/0.log" Jun 15 03:54:23.826250 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:23.826221 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5799f64c8f-fk5zd_df113ca2-256b-46fb-aec0-8693e938e085/manager/0.log" Jun 15 03:54:25.357035 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:25.357003 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-vf427_22016020-a0bf-418f-b15d-3c19de2410f5/manager/0.log" Jun 15 03:54:25.675956 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:25.675877 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-855sx_32431758-698e-4d27-b975-2dade945af3a/registry-server/0.log" Jun 15 03:54:25.790587 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:25.790552 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-pkd5m_886c9b2a-efc4-48c2-8210-64cc60cfc634/manager/0.log" Jun 15 03:54:25.900939 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:25.900914 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-lcq9d_a8f0d08c-5d9f-4a46-a402-aa7d027d0e8c/limitador/0.log" Jun 15 03:54:26.341814 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:26.341777 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795fk9ft_0afa077d-107f-4f3f-a06d-455b8d8d0a15/istio-proxy/0.log" Jun 15 03:54:26.770642 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:26.770611 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-5nd2z_363748fd-18d7-470c-9080-1bca3d60beec/istio-proxy/1.log" Jun 15 03:54:27.532756 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:27.532726 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-zb8mb_169a22b8-615c-4c60-ae38-a0afca76134d/main/0.log" Jun 15 03:54:27.541139 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:27.541115 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-zb8mb_169a22b8-615c-4c60-ae38-a0afca76134d/storage-initializer/0.log" Jun 15 03:54:27.758288 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:27.758257 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t_23d9d928-d3f5-42a5-b685-d4613e335d9e/storage-initializer/0.log" Jun 15 03:54:27.764500 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:27.764424 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cszc5t_23d9d928-d3f5-42a5-b685-d4613e335d9e/main/0.log" Jun 15 03:54:27.988138 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:27.988109 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-25qdw_80456458-9f58-40fc-8c04-1779d22b34d3/storage-initializer/0.log" Jun 15 03:54:27.994780 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:27.994750 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-25qdw_80456458-9f58-40fc-8c04-1779d22b34d3/main/0.log" Jun 15 03:54:32.324632 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:32.324594 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t4t9s/must-gather-4xppv"] Jun 15 03:54:32.325045 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:32.324968 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f820240-f63b-40ff-b046-dea2937568aa" containerName="istio-proxy" Jun 15 03:54:32.325045 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:32.324982 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f820240-f63b-40ff-b046-dea2937568aa" containerName="istio-proxy" Jun 15 03:54:32.325136 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:32.325059 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f820240-f63b-40ff-b046-dea2937568aa" containerName="istio-proxy" Jun 15 03:54:32.328132 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:32.328115 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t4t9s/must-gather-4xppv" Jun 15 03:54:32.331218 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:32.331196 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t4t9s\"/\"kube-root-ca.crt\"" Jun 15 03:54:32.331506 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:32.331486 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-t4t9s\"/\"default-dockercfg-n4l8b\"" Jun 15 03:54:32.331619 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:32.331530 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t4t9s\"/\"openshift-service-ca.crt\"" Jun 15 03:54:32.351156 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:32.351133 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t4t9s/must-gather-4xppv"] Jun 15 03:54:32.453457 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:32.453430 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dfgv\" (UniqueName: \"kubernetes.io/projected/87a517f3-bad4-4b37-be0c-5e6b0e5d80ba-kube-api-access-7dfgv\") pod \"must-gather-4xppv\" (UID: \"87a517f3-bad4-4b37-be0c-5e6b0e5d80ba\") " pod="openshift-must-gather-t4t9s/must-gather-4xppv" Jun 15 03:54:32.453575 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:32.453532 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/87a517f3-bad4-4b37-be0c-5e6b0e5d80ba-must-gather-output\") pod \"must-gather-4xppv\" (UID: \"87a517f3-bad4-4b37-be0c-5e6b0e5d80ba\") " pod="openshift-must-gather-t4t9s/must-gather-4xppv" Jun 15 03:54:32.554441 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:32.554418 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/87a517f3-bad4-4b37-be0c-5e6b0e5d80ba-must-gather-output\") pod \"must-gather-4xppv\" (UID: \"87a517f3-bad4-4b37-be0c-5e6b0e5d80ba\") " pod="openshift-must-gather-t4t9s/must-gather-4xppv" Jun 15 03:54:32.554559 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:32.554477 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dfgv\" (UniqueName: \"kubernetes.io/projected/87a517f3-bad4-4b37-be0c-5e6b0e5d80ba-kube-api-access-7dfgv\") pod \"must-gather-4xppv\" (UID: \"87a517f3-bad4-4b37-be0c-5e6b0e5d80ba\") " pod="openshift-must-gather-t4t9s/must-gather-4xppv" Jun 15 03:54:32.554761 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:32.554737 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/87a517f3-bad4-4b37-be0c-5e6b0e5d80ba-must-gather-output\") pod \"must-gather-4xppv\" (UID: \"87a517f3-bad4-4b37-be0c-5e6b0e5d80ba\") " pod="openshift-must-gather-t4t9s/must-gather-4xppv" Jun 15 03:54:32.568030 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:32.568010 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dfgv\" (UniqueName: \"kubernetes.io/projected/87a517f3-bad4-4b37-be0c-5e6b0e5d80ba-kube-api-access-7dfgv\") pod \"must-gather-4xppv\" (UID: \"87a517f3-bad4-4b37-be0c-5e6b0e5d80ba\") " pod="openshift-must-gather-t4t9s/must-gather-4xppv" Jun 15 03:54:32.637641 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:32.637615 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t4t9s/must-gather-4xppv" Jun 15 03:54:32.772995 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:32.772969 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t4t9s/must-gather-4xppv"] Jun 15 03:54:32.774846 ip-10-0-143-185 kubenswrapper[2562]: W0615 03:54:32.774812 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87a517f3_bad4_4b37_be0c_5e6b0e5d80ba.slice/crio-056a35965cf0de80f9852d3761b368e7331be924dd70c3629f59e33a9d0eb1ba WatchSource:0}: Error finding container 056a35965cf0de80f9852d3761b368e7331be924dd70c3629f59e33a9d0eb1ba: Status 404 returned error can't find the container with id 056a35965cf0de80f9852d3761b368e7331be924dd70c3629f59e33a9d0eb1ba Jun 15 03:54:32.776882 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:32.776864 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 15 03:54:33.244788 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:33.244757 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t4t9s/must-gather-4xppv" event={"ID":"87a517f3-bad4-4b37-be0c-5e6b0e5d80ba","Type":"ContainerStarted","Data":"056a35965cf0de80f9852d3761b368e7331be924dd70c3629f59e33a9d0eb1ba"} Jun 15 03:54:34.251732 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:34.251690 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t4t9s/must-gather-4xppv" event={"ID":"87a517f3-bad4-4b37-be0c-5e6b0e5d80ba","Type":"ContainerStarted","Data":"103b38fe0d4828d5e99f41cd393b1e8c8d253dd4a53d7186337c68046e693f2c"} Jun 15 03:54:34.251732 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:34.251736 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t4t9s/must-gather-4xppv" event={"ID":"87a517f3-bad4-4b37-be0c-5e6b0e5d80ba","Type":"ContainerStarted","Data":"5f7da02352df0d17eae4408fcabdca42773f507c2fd6848bccc17696177629b4"} Jun 15 03:54:34.265914 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:34.265855 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-t4t9s/must-gather-4xppv" podStartSLOduration=1.607618827 podStartE2EDuration="2.265820317s" totalImagesPullingTime="658.20149ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-15 03:54:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:54:34.265142625 +0000 UTC m=+2742.576281631" watchObservedRunningTime="2026-06-15 03:54:34.265820317 +0000 UTC m=+2742.576959308" Jun 15 03:54:35.228573 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:35.228544 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mbtvm_a3db17e5-7acd-462b-a6f2-b11617619628/global-pull-secret-syncer/0.log" Jun 15 03:54:35.368739 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:35.368706 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-s4gh2_0a161380-ecd7-4473-9cc2-e3041ad5571d/konnectivity-agent/0.log" Jun 15 03:54:35.436652 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:35.436620 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-185.ec2.internal_a43d74820b080a54928ef60318b8a3ec/haproxy/0.log" Jun 15 03:54:40.632206 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:40.632173 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-vf427_22016020-a0bf-418f-b15d-3c19de2410f5/manager/0.log" Jun 15 03:54:40.732091 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:40.731911 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-855sx_32431758-698e-4d27-b975-2dade945af3a/registry-server/0.log" Jun 15 03:54:40.863225 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:40.863194 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-pkd5m_886c9b2a-efc4-48c2-8210-64cc60cfc634/manager/0.log" Jun 15 03:54:40.890031 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:40.889947 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-lcq9d_a8f0d08c-5d9f-4a46-a402-aa7d027d0e8c/limitador/0.log" Jun 15 03:54:42.915280 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:42.915245 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-589576dfc7-pzl7w_455c4a83-df24-49bb-b1a2-f427ed4a795f/monitoring-plugin/0.log" Jun 15 03:54:43.111347 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:43.111309 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hmqkw_423a0d92-fe2b-4a84-b581-1bec5edad495/node-exporter/0.log" Jun 15 03:54:43.136004 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:43.135971 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hmqkw_423a0d92-fe2b-4a84-b581-1bec5edad495/kube-rbac-proxy/0.log" Jun 15 03:54:43.158293 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:43.158104 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hmqkw_423a0d92-fe2b-4a84-b581-1bec5edad495/init-textfile/0.log" Jun 15 03:54:43.189946 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:43.189869 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6b8d965dd5-tnhbx_63801ac3-c391-4e2f-93ca-8f0a495d672d/kube-rbac-proxy-main/0.log" Jun 15 03:54:43.211058 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:43.211026 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6b8d965dd5-tnhbx_63801ac3-c391-4e2f-93ca-8f0a495d672d/kube-rbac-proxy-self/0.log" Jun 15 03:54:43.238600 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:43.238567 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6b8d965dd5-tnhbx_63801ac3-c391-4e2f-93ca-8f0a495d672d/openshift-state-metrics/0.log" Jun 15 03:54:43.307392 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:43.307351 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ba6152df-a718-417e-b907-540d1ba9079a/prometheus/0.log" Jun 15 03:54:43.328010 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:43.327950 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ba6152df-a718-417e-b907-540d1ba9079a/config-reloader/0.log" Jun 15 03:54:43.349583 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:43.349550 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ba6152df-a718-417e-b907-540d1ba9079a/thanos-sidecar/0.log" Jun 15 03:54:43.369726 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:43.369641 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ba6152df-a718-417e-b907-540d1ba9079a/kube-rbac-proxy-web/0.log" Jun 15 03:54:43.395674 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:43.395643 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ba6152df-a718-417e-b907-540d1ba9079a/kube-rbac-proxy/0.log" Jun 15 03:54:43.417851 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:43.417804 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ba6152df-a718-417e-b907-540d1ba9079a/kube-rbac-proxy-thanos/0.log" Jun 15 03:54:43.438224 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:43.438190 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ba6152df-a718-417e-b907-540d1ba9079a/init-config-reloader/0.log" Jun 15 03:54:46.410730 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:46.410690 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-694c955b9-xqnxf_3d282f55-36e3-45f9-9a11-505ef16fc6c6/download-server/0.log" Jun 15 03:54:47.526518 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:47.526476 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c"] Jun 15 03:54:47.535893 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:47.535863 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" Jun 15 03:54:47.536670 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:47.536644 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c"] Jun 15 03:54:47.621495 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:47.621451 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/30947264-5731-4f82-9fe2-74dc87ebd9a5-proc\") pod \"perf-node-gather-daemonset-wtp4c\" (UID: \"30947264-5731-4f82-9fe2-74dc87ebd9a5\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" Jun 15 03:54:47.621495 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:47.621497 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/30947264-5731-4f82-9fe2-74dc87ebd9a5-podres\") pod \"perf-node-gather-daemonset-wtp4c\" (UID: \"30947264-5731-4f82-9fe2-74dc87ebd9a5\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" Jun 15 03:54:47.621749 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:47.621529 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30947264-5731-4f82-9fe2-74dc87ebd9a5-sys\") pod \"perf-node-gather-daemonset-wtp4c\" (UID: \"30947264-5731-4f82-9fe2-74dc87ebd9a5\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" Jun 15 03:54:47.621749 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:47.621553 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30947264-5731-4f82-9fe2-74dc87ebd9a5-lib-modules\") pod \"perf-node-gather-daemonset-wtp4c\" (UID: \"30947264-5731-4f82-9fe2-74dc87ebd9a5\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" Jun 15 03:54:47.621749 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:47.621634 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qr95\" (UniqueName: \"kubernetes.io/projected/30947264-5731-4f82-9fe2-74dc87ebd9a5-kube-api-access-8qr95\") pod \"perf-node-gather-daemonset-wtp4c\" (UID: \"30947264-5731-4f82-9fe2-74dc87ebd9a5\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" Jun 15 03:54:47.723982 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:47.723011 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/30947264-5731-4f82-9fe2-74dc87ebd9a5-proc\") pod \"perf-node-gather-daemonset-wtp4c\" (UID: \"30947264-5731-4f82-9fe2-74dc87ebd9a5\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" Jun 15 03:54:47.723982 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:47.723062 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/30947264-5731-4f82-9fe2-74dc87ebd9a5-podres\") pod \"perf-node-gather-daemonset-wtp4c\" (UID: \"30947264-5731-4f82-9fe2-74dc87ebd9a5\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" Jun 15 03:54:47.723982 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:47.723093 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30947264-5731-4f82-9fe2-74dc87ebd9a5-sys\") pod \"perf-node-gather-daemonset-wtp4c\" (UID: \"30947264-5731-4f82-9fe2-74dc87ebd9a5\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" Jun 15 03:54:47.723982 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:47.723119 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30947264-5731-4f82-9fe2-74dc87ebd9a5-lib-modules\") pod \"perf-node-gather-daemonset-wtp4c\" (UID: \"30947264-5731-4f82-9fe2-74dc87ebd9a5\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" Jun 15 03:54:47.723982 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:47.723197 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qr95\" (UniqueName: \"kubernetes.io/projected/30947264-5731-4f82-9fe2-74dc87ebd9a5-kube-api-access-8qr95\") pod \"perf-node-gather-daemonset-wtp4c\" (UID: \"30947264-5731-4f82-9fe2-74dc87ebd9a5\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" Jun 15 03:54:47.723982 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:47.723648 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/30947264-5731-4f82-9fe2-74dc87ebd9a5-proc\") pod \"perf-node-gather-daemonset-wtp4c\" (UID: \"30947264-5731-4f82-9fe2-74dc87ebd9a5\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" Jun 15 03:54:47.723982 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:47.723781 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/30947264-5731-4f82-9fe2-74dc87ebd9a5-podres\") pod \"perf-node-gather-daemonset-wtp4c\" (UID: \"30947264-5731-4f82-9fe2-74dc87ebd9a5\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" Jun 15 03:54:47.723982 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:47.723861 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30947264-5731-4f82-9fe2-74dc87ebd9a5-sys\") pod \"perf-node-gather-daemonset-wtp4c\" (UID: \"30947264-5731-4f82-9fe2-74dc87ebd9a5\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" Jun 15 03:54:47.723982 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:47.723952 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30947264-5731-4f82-9fe2-74dc87ebd9a5-lib-modules\") pod \"perf-node-gather-daemonset-wtp4c\" (UID: \"30947264-5731-4f82-9fe2-74dc87ebd9a5\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" Jun 15 03:54:47.734007 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:47.733976 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qr95\" (UniqueName: \"kubernetes.io/projected/30947264-5731-4f82-9fe2-74dc87ebd9a5-kube-api-access-8qr95\") pod \"perf-node-gather-daemonset-wtp4c\" (UID: \"30947264-5731-4f82-9fe2-74dc87ebd9a5\") " pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" Jun 15 03:54:47.850293 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:47.850002 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" Jun 15 03:54:48.031098 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:48.030370 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c"] Jun 15 03:54:48.345561 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:48.345415 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" event={"ID":"30947264-5731-4f82-9fe2-74dc87ebd9a5","Type":"ContainerStarted","Data":"b7d2d0b88986d2d6cb150db0eedf4ab502b60d2197fc4fd457ec9a72ae667939"} Jun 15 03:54:48.345561 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:48.345486 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" event={"ID":"30947264-5731-4f82-9fe2-74dc87ebd9a5","Type":"ContainerStarted","Data":"9f37b9ddb1a0b955b0abb1b876ae109ad3ad92925873dd79a8d8ceab898c13c5"} Jun 15 03:54:48.345561 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:48.345528 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" Jun 15 03:54:48.363503 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:48.363445 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" podStartSLOduration=1.363425555 podStartE2EDuration="1.363425555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-15 03:54:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-15 03:54:48.361157042 +0000 UTC m=+2756.672296033" watchObservedRunningTime="2026-06-15 03:54:48.363425555 +0000 UTC m=+2756.674564549" Jun 15 03:54:48.470980 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:48.470951 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nb5km_167a4bfe-f33d-4a1e-878c-8faebc6f61e7/dns/0.log" Jun 15 03:54:48.495280 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:48.495245 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nb5km_167a4bfe-f33d-4a1e-878c-8faebc6f61e7/kube-rbac-proxy/0.log" Jun 15 03:54:48.563931 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:48.563896 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lj57j_ff9255c7-dc08-4be9-9087-b749284c6358/dns-node-resolver/0.log" Jun 15 03:54:49.218746 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:49.218705 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xhccz_7fe3e8d4-b5da-4511-a7f8-f28596c5f336/node-ca/0.log" Jun 15 03:54:50.208343 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:50.208311 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795fk9ft_0afa077d-107f-4f3f-a06d-455b8d8d0a15/istio-proxy/0.log" Jun 15 03:54:50.367040 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:50.366996 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-5nd2z_363748fd-18d7-470c-9080-1bca3d60beec/istio-proxy/0.log" Jun 15 03:54:50.403470 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:50.403436 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-5nd2z_363748fd-18d7-470c-9080-1bca3d60beec/istio-proxy/1.log" Jun 15 03:54:51.152743 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:51.152682 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p9h4n_f61b9993-27b1-4a1d-b7bc-632802832528/serve-healthcheck-canary/0.log" Jun 15 03:54:51.673292 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:51.673254 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5bbd86d6bd-dswhg_06bfd370-abe8-4b3a-a82c-41e7d0ade9c7/insights-operator/0.log" Jun 15 03:54:51.676260 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:51.676235 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5bbd86d6bd-dswhg_06bfd370-abe8-4b3a-a82c-41e7d0ade9c7/insights-operator/1.log" Jun 15 03:54:51.695905 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:51.695885 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pmfbb_b0fe013e-5d5e-441f-bda2-50fc7a5c2140/kube-rbac-proxy/0.log" Jun 15 03:54:51.716827 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:51.716807 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pmfbb_b0fe013e-5d5e-441f-bda2-50fc7a5c2140/exporter/0.log" Jun 15 03:54:51.738599 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:51.738569 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pmfbb_b0fe013e-5d5e-441f-bda2-50fc7a5c2140/extractor/0.log" Jun 15 03:54:53.999065 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:53.999026 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-694cb6497d-45zn5_277599cb-8940-479c-a249-5244193660d4/nginx/0.log" Jun 15 03:54:54.368074 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:54.368043 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-t4t9s/perf-node-gather-daemonset-wtp4c" Jun 15 03:54:54.465740 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:54.465679 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5799f64c8f-fk5zd_df113ca2-256b-46fb-aec0-8693e938e085/manager/0.log" Jun 15 03:54:54.484694 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:54.484659 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5799f64c8f-fk5zd_df113ca2-256b-46fb-aec0-8693e938e085/copy-manifests/0.log" Jun 15 03:54:56.089569 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:54:56.089540 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6b4969d8b8-bk8p9_790c85c2-7984-4567-b4a1-1083ad9b3b94/manager/0.log" Jun 15 03:55:00.034481 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:55:00.034455 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795fk9ft_0afa077d-107f-4f3f-a06d-455b8d8d0a15/istio-proxy/0.log" Jun 15 03:55:00.144448 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:55:00.144409 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-5nd2z_363748fd-18d7-470c-9080-1bca3d60beec/istio-proxy/0.log" Jun 15 03:55:00.189380 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:55:00.189347 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-5nd2z_363748fd-18d7-470c-9080-1bca3d60beec/istio-proxy/1.log" Jun 15 03:55:01.414858 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:55:01.414799 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795fk9ft_0afa077d-107f-4f3f-a06d-455b8d8d0a15/istio-proxy/0.log" Jun 15 03:55:01.529597 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:55:01.529564 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-5nd2z_363748fd-18d7-470c-9080-1bca3d60beec/istio-proxy/0.log" Jun 15 03:55:01.573370 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:55:01.573316 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-5nd2z_363748fd-18d7-470c-9080-1bca3d60beec/istio-proxy/1.log" Jun 15 03:55:03.180013 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:55:03.179980 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn582_28f12465-f135-4586-a19f-98430d1f26b0/kube-multus-additional-cni-plugins/0.log" Jun 15 03:55:03.201447 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:55:03.201417 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn582_28f12465-f135-4586-a19f-98430d1f26b0/egress-router-binary-copy/0.log" Jun 15 03:55:03.224924 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:55:03.224819 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn582_28f12465-f135-4586-a19f-98430d1f26b0/cni-plugins/0.log" Jun 15 03:55:03.246953 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:55:03.246922 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn582_28f12465-f135-4586-a19f-98430d1f26b0/bond-cni-plugin/0.log" Jun 15 03:55:03.269785 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:55:03.269758 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn582_28f12465-f135-4586-a19f-98430d1f26b0/routeoverride-cni/0.log" Jun 15 03:55:03.295696 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:55:03.295664 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn582_28f12465-f135-4586-a19f-98430d1f26b0/whereabouts-cni-bincopy/0.log" Jun 15 03:55:03.316577 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:55:03.316552 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pn582_28f12465-f135-4586-a19f-98430d1f26b0/whereabouts-cni/0.log" Jun 15 03:55:03.350235 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:55:03.350206 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795fk9ft_0afa077d-107f-4f3f-a06d-455b8d8d0a15/istio-proxy/0.log" Jun 15 03:55:03.448479 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:55:03.448382 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-5nd2z_363748fd-18d7-470c-9080-1bca3d60beec/istio-proxy/0.log" Jun 15 03:55:03.487679 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:55:03.487647 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-5nd2z_363748fd-18d7-470c-9080-1bca3d60beec/istio-proxy/1.log" Jun 15 03:55:03.695977 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:55:03.695943 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-tpvc8_d75956ab-4288-4724-a60e-f55e2cb6c932/kube-multus/0.log" Jun 15 03:55:03.792411 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:55:03.792341 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vz9g7_c85b7d1f-4d4b-4fdd-9427-d1c25942aebe/network-metrics-daemon/0.log" Jun 15 03:55:03.812080 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:55:03.812054 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vz9g7_c85b7d1f-4d4b-4fdd-9427-d1c25942aebe/kube-rbac-proxy/0.log" Jun 15 03:55:05.051730 ip-10-0-143-185 kubenswrapper[2562]: I0615 03:55:05.051688 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9r9mn_789deb33-3a27-4d7f-881f-31f889f46e0e/ovn-controller/0.log"