Jul 23 12:07:58.892279 ip-10-0-138-37 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 23 12:07:58.892289 ip-10-0-138-37 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 23 12:07:58.892299 ip-10-0-138-37 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 23 12:07:58.892626 ip-10-0-138-37 systemd[1]: Failed to start Kubernetes Kubelet. Jul 23 12:08:09.027571 ip-10-0-138-37 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 23 12:08:09.027594 ip-10-0-138-37 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 624b414f2ceb4c2fb87d7c7c319dcfa6 -- Jul 23 12:09:47.110616 ip-10-0-138-37 systemd[1]: Starting Kubernetes Kubelet... Jul 23 12:09:47.592267 ip-10-0-138-37 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 12:09:47.592267 ip-10-0-138-37 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 23 12:09:47.592267 ip-10-0-138-37 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 12:09:47.592267 ip-10-0-138-37 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 23 12:09:47.592267 ip-10-0-138-37 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 12:09:47.592267 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.547990 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 23 12:09:47.592267 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554418 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 12:09:47.592267 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554434 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 12:09:47.592267 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554438 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 12:09:47.592267 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554441 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 12:09:47.592267 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554445 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 12:09:47.592267 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554449 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 12:09:47.592267 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554454 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 12:09:47.594559 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554457 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 12:09:47.594559 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554460 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 12:09:47.594559 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554463 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 12:09:47.594559 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554466 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 12:09:47.594559 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554469 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 12:09:47.594559 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554472 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 12:09:47.594559 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554475 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 12:09:47.594559 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554478 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 12:09:47.594559 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554481 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 12:09:47.594559 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554483 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 12:09:47.594559 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554486 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 12:09:47.594559 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554494 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 12:09:47.594559 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554497 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 12:09:47.594559 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554500 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 12:09:47.594559 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554503 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 12:09:47.594559 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554505 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 12:09:47.594559 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554508 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 12:09:47.594559 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554511 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 12:09:47.594559 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554513 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 12:09:47.594559 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554516 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 12:09:47.596848 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554518 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 12:09:47.596848 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554521 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 12:09:47.596848 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554524 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 12:09:47.596848 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554528 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 12:09:47.596848 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554531 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 12:09:47.596848 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554534 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 12:09:47.596848 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554536 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 12:09:47.596848 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554539 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 12:09:47.596848 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554542 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 12:09:47.596848 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554546 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 12:09:47.596848 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554549 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 12:09:47.596848 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554551 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 12:09:47.596848 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554554 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 12:09:47.596848 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554556 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 12:09:47.596848 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554559 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 12:09:47.596848 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554562 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 12:09:47.596848 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554564 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 12:09:47.596848 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554567 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 12:09:47.596848 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554569 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 12:09:47.598744 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554572 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 12:09:47.598744 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554576 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 12:09:47.598744 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554580 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 12:09:47.598744 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554583 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 12:09:47.598744 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554585 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 12:09:47.598744 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554589 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 12:09:47.598744 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554592 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 12:09:47.598744 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554594 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 12:09:47.598744 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554597 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 12:09:47.598744 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554599 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 12:09:47.598744 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554602 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 12:09:47.598744 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554605 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 12:09:47.598744 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554607 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 12:09:47.598744 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554610 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 12:09:47.598744 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554613 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 12:09:47.598744 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554615 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 12:09:47.598744 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554618 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 12:09:47.598744 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554621 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 12:09:47.598744 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554623 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 12:09:47.612333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554626 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 12:09:47.612333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554628 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 12:09:47.612333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554631 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 12:09:47.612333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554633 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 12:09:47.612333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554636 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 12:09:47.612333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554639 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 12:09:47.612333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554642 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 12:09:47.612333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554644 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 12:09:47.612333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554647 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 12:09:47.612333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554649 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 12:09:47.612333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554651 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 12:09:47.612333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554654 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 12:09:47.612333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554657 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 12:09:47.612333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554660 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 12:09:47.612333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554665 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 12:09:47.612333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554668 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 12:09:47.612333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554670 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 12:09:47.612333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554673 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 12:09:47.612333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554676 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 12:09:47.612333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554678 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 12:09:47.614230 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554681 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 12:09:47.614230 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554684 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 12:09:47.614230 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554686 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 12:09:47.614230 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554690 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 12:09:47.614230 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554693 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 12:09:47.614230 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554697 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 12:09:47.614230 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554700 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 12:09:47.614230 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554703 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 12:09:47.614230 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554706 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 12:09:47.614230 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554709 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 12:09:47.614230 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554712 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 12:09:47.614230 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554714 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 12:09:47.614230 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554717 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 12:09:47.614230 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.554719 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 12:09:47.614230 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556484 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 12:09:47.614230 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556502 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 12:09:47.614230 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556507 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 12:09:47.614230 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556510 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 12:09:47.614230 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556516 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 12:09:47.614230 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556519 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 12:09:47.616636 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556522 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 12:09:47.616636 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556525 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 12:09:47.616636 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556528 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 12:09:47.616636 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556532 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 12:09:47.616636 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556536 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 12:09:47.616636 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556539 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 12:09:47.616636 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556542 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 12:09:47.616636 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556545 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 12:09:47.616636 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556547 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 12:09:47.616636 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556550 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 12:09:47.616636 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556553 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 12:09:47.616636 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556556 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 12:09:47.616636 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556558 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 12:09:47.616636 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556561 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 12:09:47.616636 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556565 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 12:09:47.616636 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556568 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 12:09:47.616636 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556571 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 12:09:47.616636 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556573 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 12:09:47.616636 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556576 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 12:09:47.616636 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556579 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 12:09:47.632055 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556581 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 12:09:47.632055 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556585 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 12:09:47.632055 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556588 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 12:09:47.632055 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556590 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 12:09:47.632055 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556593 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 12:09:47.632055 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556596 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 12:09:47.632055 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556599 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 12:09:47.632055 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556602 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 12:09:47.632055 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556605 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 12:09:47.632055 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556607 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 12:09:47.632055 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556610 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 12:09:47.632055 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556613 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 12:09:47.632055 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556615 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 12:09:47.632055 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556618 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 12:09:47.632055 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556622 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 12:09:47.632055 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556625 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 12:09:47.632055 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556628 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 12:09:47.632055 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556630 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 12:09:47.632055 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556633 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 12:09:47.632055 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556635 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 12:09:47.622219 ip-10-0-138-37 systemd[1]: Started Kubernetes Kubelet. Jul 23 12:09:47.634035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556638 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 12:09:47.634035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556640 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 12:09:47.634035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556643 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 12:09:47.634035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556645 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 12:09:47.634035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556649 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 12:09:47.634035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556652 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 12:09:47.634035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556655 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 12:09:47.634035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556658 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 12:09:47.634035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556661 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 12:09:47.634035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556663 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 12:09:47.634035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556666 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 12:09:47.634035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556669 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 12:09:47.634035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556671 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 12:09:47.634035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556674 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 12:09:47.634035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556677 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 12:09:47.634035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556679 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 12:09:47.634035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556682 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 12:09:47.634035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556686 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 12:09:47.634035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556689 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 12:09:47.634035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556692 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 12:09:47.636283 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556694 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 12:09:47.636283 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556697 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 12:09:47.636283 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556699 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 12:09:47.636283 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556702 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 12:09:47.636283 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556705 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 12:09:47.636283 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556708 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 12:09:47.636283 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556710 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 12:09:47.636283 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556714 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 12:09:47.636283 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556719 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 12:09:47.636283 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556722 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 12:09:47.636283 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556725 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 12:09:47.636283 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556728 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 12:09:47.636283 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556731 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 12:09:47.636283 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556735 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 12:09:47.636283 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556738 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 12:09:47.636283 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556741 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 12:09:47.636283 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556744 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 12:09:47.636283 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556747 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 12:09:47.636283 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556750 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 12:09:47.652597 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556753 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 12:09:47.652597 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556755 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 12:09:47.652597 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556758 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 12:09:47.652597 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556760 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 12:09:47.652597 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556763 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 12:09:47.652597 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556767 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 12:09:47.652597 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556770 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 12:09:47.652597 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556773 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 12:09:47.652597 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556776 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 12:09:47.652597 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556779 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 12:09:47.652597 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556782 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 12:09:47.652597 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556785 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 12:09:47.652597 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556788 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 12:09:47.652597 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.556791 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 12:09:47.652597 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557178 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 12:09:47.652597 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557183 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 12:09:47.652597 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557189 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 12:09:47.652597 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557192 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 12:09:47.652597 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557195 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 12:09:47.655910 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557197 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 12:09:47.655910 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557200 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 12:09:47.655910 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557203 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 12:09:47.655910 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557206 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 12:09:47.655910 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557208 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 12:09:47.655910 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557211 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 12:09:47.655910 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557214 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 12:09:47.655910 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557216 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 12:09:47.655910 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557219 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 12:09:47.655910 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557221 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 12:09:47.655910 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557224 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 12:09:47.655910 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557227 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 12:09:47.655910 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557230 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 12:09:47.655910 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557233 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 12:09:47.655910 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557236 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 12:09:47.655910 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557238 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 12:09:47.655910 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557241 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 12:09:47.655910 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557243 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 12:09:47.655910 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557246 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 12:09:47.655910 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557249 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 12:09:47.669716 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557251 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 12:09:47.669716 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557254 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 12:09:47.669716 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557256 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 12:09:47.669716 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557259 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 12:09:47.669716 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557261 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 12:09:47.669716 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557265 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 12:09:47.669716 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557267 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 12:09:47.669716 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557270 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 12:09:47.669716 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557273 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 12:09:47.669716 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557279 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 12:09:47.669716 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557282 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 12:09:47.669716 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557285 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 12:09:47.669716 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557287 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 12:09:47.669716 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557290 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 12:09:47.669716 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557292 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 12:09:47.669716 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557295 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 12:09:47.669716 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557298 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 12:09:47.669716 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557300 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 12:09:47.669716 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557303 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 12:09:47.669716 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557306 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 12:09:47.702501 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557308 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 12:09:47.702501 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557311 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 12:09:47.702501 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557313 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 12:09:47.702501 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557336 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 12:09:47.702501 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557340 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 12:09:47.702501 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557342 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 12:09:47.702501 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557345 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 12:09:47.702501 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557348 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 12:09:47.702501 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557351 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 12:09:47.702501 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557353 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 12:09:47.702501 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557356 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 12:09:47.702501 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557358 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 12:09:47.702501 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557361 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 12:09:47.702501 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557364 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 12:09:47.702501 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557367 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 12:09:47.702501 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557369 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 12:09:47.702501 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557372 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 12:09:47.702501 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557375 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 12:09:47.702501 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557377 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 12:09:47.702501 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557380 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 12:09:47.837092 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557383 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 12:09:47.837092 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557388 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 12:09:47.837092 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557391 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 12:09:47.837092 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557393 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 12:09:47.837092 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557396 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 12:09:47.837092 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557398 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 12:09:47.837092 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557401 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 12:09:47.837092 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557404 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 12:09:47.837092 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557406 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 12:09:47.837092 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557411 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 12:09:47.837092 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557414 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 12:09:47.837092 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557418 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 12:09:47.837092 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557421 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 12:09:47.837092 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557423 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 12:09:47.837092 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557426 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 12:09:47.837092 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557429 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 12:09:47.837092 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557431 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 12:09:47.837092 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557435 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 12:09:47.837092 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557438 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557441 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557443 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557446 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557448 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557451 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557455 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557459 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557463 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557466 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557469 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557471 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557474 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557476 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557479 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.557484 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557562 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557570 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557583 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557587 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557591 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 23 12:09:47.838100 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557594 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557598 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557603 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557606 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557609 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557612 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557616 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557619 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557622 2578 flags.go:64] FLAG: --cgroup-root="" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557625 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557628 2578 flags.go:64] FLAG: --client-ca-file="" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557631 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557634 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557640 2578 flags.go:64] FLAG: --cluster-domain="" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557643 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557647 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557650 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557653 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557657 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557660 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557663 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557666 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557669 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557672 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 23 12:09:47.838912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557675 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557678 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557683 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557687 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557690 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557693 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557696 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557699 2578 flags.go:64] FLAG: --enable-server="true" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557702 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557707 2578 flags.go:64] FLAG: --event-burst="100" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557710 2578 flags.go:64] FLAG: --event-qps="50" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557713 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557716 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557719 2578 flags.go:64] FLAG: --eviction-hard="" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557723 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557726 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557729 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557732 2578 flags.go:64] FLAG: --eviction-soft="" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557735 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557738 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557741 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557744 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557747 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557750 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557753 2578 flags.go:64] FLAG: --feature-gates="" Jul 23 12:09:47.839771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557757 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557760 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557762 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557766 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557768 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557772 2578 flags.go:64] FLAG: --help="false" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557774 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-138-37.ec2.internal" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557778 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557780 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557786 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557789 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557793 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557795 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557798 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557801 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557804 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557807 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557810 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557813 2578 flags.go:64] FLAG: --kube-reserved="" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557816 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557819 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557822 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557825 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557827 2578 flags.go:64] FLAG: --lock-file="" Jul 23 12:09:47.840594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557830 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557833 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557836 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557842 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557845 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557848 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557851 2578 flags.go:64] FLAG: --logging-format="text" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557854 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557857 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557860 2578 flags.go:64] FLAG: --manifest-url="" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557862 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557866 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557869 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557873 2578 flags.go:64] FLAG: --max-pods="110" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557876 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557879 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557882 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557887 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557890 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557894 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557897 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557904 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557907 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557910 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 23 12:09:47.843901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557913 2578 flags.go:64] FLAG: --pod-cidr="" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557916 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557922 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557925 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557928 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557931 2578 flags.go:64] FLAG: --port="10250" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557934 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557937 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-073bd416155feedbc" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557940 2578 flags.go:64] FLAG: --qos-reserved="" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557943 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557946 2578 flags.go:64] FLAG: --register-node="true" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557949 2578 flags.go:64] FLAG: --register-with-taints="" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557953 2578 flags.go:64] FLAG: --registry-burst="10" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557956 2578 flags.go:64] FLAG: --registry-qps="5" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557959 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557962 2578 flags.go:64] FLAG: --reserved-memory="" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557966 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557969 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557972 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557975 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557978 2578 flags.go:64] FLAG: --runonce="false" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557980 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557983 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557986 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557989 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 23 12:09:47.844666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557993 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.557997 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558000 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558003 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558007 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558010 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558012 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558015 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558018 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558022 2578 flags.go:64] FLAG: --system-cgroups="" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558025 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558030 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558033 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558035 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558041 2578 flags.go:64] FLAG: --tls-min-version="" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558044 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558046 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558049 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558052 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558055 2578 flags.go:64] FLAG: --v="2" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558062 2578 flags.go:64] FLAG: --version="false" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558066 2578 flags.go:64] FLAG: --vmodule="" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558070 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.558073 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558186 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 12:09:47.846080 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558189 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 12:09:47.846799 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558192 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 12:09:47.846799 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558195 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 12:09:47.846799 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558198 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 12:09:47.846799 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558200 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 12:09:47.846799 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558203 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 12:09:47.846799 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558205 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 12:09:47.846799 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558210 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 12:09:47.846799 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558214 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 12:09:47.846799 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558217 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 12:09:47.846799 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558219 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 12:09:47.846799 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558222 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 12:09:47.846799 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558225 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 12:09:47.846799 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558227 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 12:09:47.846799 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558230 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 12:09:47.846799 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558233 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 12:09:47.846799 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558236 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 12:09:47.846799 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558238 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 12:09:47.846799 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558241 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 12:09:47.846799 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558243 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 12:09:47.846799 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558246 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 12:09:47.847361 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558248 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 12:09:47.847361 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558251 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 12:09:47.847361 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558253 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 12:09:47.847361 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558256 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 12:09:47.847361 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558259 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 12:09:47.847361 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558261 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 12:09:47.847361 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558264 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 12:09:47.847361 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558266 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 12:09:47.847361 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558269 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 12:09:47.847361 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558272 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 12:09:47.847361 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558274 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 12:09:47.847361 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558277 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 12:09:47.847361 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558279 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 12:09:47.847361 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558282 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 12:09:47.847361 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558284 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 12:09:47.847361 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558287 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 12:09:47.847361 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558290 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 12:09:47.847361 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558294 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 12:09:47.847361 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558298 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 12:09:47.975504 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558303 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 12:09:47.975504 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558306 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 12:09:47.975504 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558309 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 12:09:47.975504 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558312 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 12:09:47.975504 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558330 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 12:09:47.975504 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558334 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 12:09:47.975504 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558337 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 12:09:47.975504 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558340 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 12:09:47.975504 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558343 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 12:09:47.975504 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558346 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 12:09:47.975504 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558348 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 12:09:47.975504 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558351 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 12:09:47.975504 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558354 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 12:09:47.975504 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558357 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 12:09:47.975504 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558359 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 12:09:47.975504 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558362 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 12:09:47.975504 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558365 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 12:09:47.975504 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558367 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 12:09:47.975504 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558370 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 12:09:47.975504 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558373 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 12:09:47.976179 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558375 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 12:09:47.976179 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558378 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 12:09:47.976179 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558381 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 12:09:47.976179 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558384 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 12:09:47.976179 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558386 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 12:09:47.976179 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558389 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 12:09:47.976179 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558391 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 12:09:47.976179 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558395 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 12:09:47.976179 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558399 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 12:09:47.976179 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558402 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 12:09:47.976179 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558405 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 12:09:47.976179 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558409 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 12:09:47.976179 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558413 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 12:09:47.976179 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558416 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 12:09:47.976179 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558418 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 12:09:47.976179 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558421 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 12:09:47.976179 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558423 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 12:09:47.976179 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558426 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 12:09:47.976179 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558429 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 12:09:47.976179 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558432 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 12:09:47.976856 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558434 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 12:09:47.976856 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558437 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 12:09:47.976856 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558439 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 12:09:47.976856 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558442 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 12:09:47.976856 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558445 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 12:09:47.976856 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558447 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 12:09:47.976856 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558450 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 12:09:47.976856 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558452 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 12:09:47.976856 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558455 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 12:09:47.976856 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558457 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 12:09:47.976856 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558460 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 12:09:47.976856 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558463 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 12:09:47.976856 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558465 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 12:09:47.976856 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558468 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 12:09:47.976856 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558472 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 12:09:47.976856 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558476 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 12:09:47.976856 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558479 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 12:09:47.976856 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.558481 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 12:09:47.977374 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.559365 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 12:09:47.977374 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.566784 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 23 12:09:47.977374 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.566803 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 23 12:09:47.977374 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.566832 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 23 12:09:47.977374 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.566840 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 23 12:09:47.977374 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566900 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 12:09:47.977374 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566905 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 12:09:47.977374 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566909 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 12:09:47.977374 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566914 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 12:09:47.977374 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566918 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 12:09:47.977374 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566921 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 12:09:47.977374 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566924 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 12:09:47.977374 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566927 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 12:09:47.977374 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566930 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 12:09:47.977374 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566933 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 12:09:47.977885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566935 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 12:09:47.977885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566938 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 12:09:47.977885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566941 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 12:09:47.977885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566943 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 12:09:47.977885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566946 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 12:09:47.977885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566949 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 12:09:47.977885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566952 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 12:09:47.977885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566954 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 12:09:47.977885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566957 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 12:09:47.977885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566959 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 12:09:47.977885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566962 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 12:09:47.977885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566965 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 12:09:47.977885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566968 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 12:09:47.977885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566970 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 12:09:47.977885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566973 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 12:09:47.977885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566975 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 12:09:47.977885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566978 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 12:09:47.977885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566981 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 12:09:47.977885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566984 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 12:09:47.977885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566987 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 12:09:47.978439 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566990 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 12:09:47.978439 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566993 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 12:09:47.978439 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566996 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 12:09:47.978439 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.566998 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 12:09:47.978439 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567000 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 12:09:47.978439 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567003 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 12:09:47.978439 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567008 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 12:09:47.978439 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567011 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 12:09:47.978439 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567014 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 12:09:47.978439 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567016 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 12:09:47.978439 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567019 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 12:09:47.978439 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567022 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 12:09:47.978439 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567026 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 12:09:47.978439 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567030 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 12:09:47.978439 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567033 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 12:09:47.978439 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567036 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 12:09:47.978439 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567039 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 12:09:47.978439 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567041 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 12:09:47.978439 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567044 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 12:09:47.979035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567047 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 12:09:47.979035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567050 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 12:09:47.979035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567052 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 12:09:47.979035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567055 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 12:09:47.979035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567057 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 12:09:47.979035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567060 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 12:09:47.979035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567064 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 12:09:47.979035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567068 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 12:09:47.979035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567071 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 12:09:47.979035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567073 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 12:09:47.979035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567078 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 12:09:47.979035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567080 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 12:09:47.979035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567083 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 12:09:47.979035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567087 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 12:09:47.979035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567089 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 12:09:47.979035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567092 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 12:09:47.979035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567094 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 12:09:47.979035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567097 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 12:09:47.979035 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567099 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 12:09:47.979560 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567103 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 12:09:47.979560 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567106 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 12:09:47.979560 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567109 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 12:09:47.979560 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567112 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 12:09:47.979560 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567114 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 12:09:47.979560 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567117 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 12:09:47.979560 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567119 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 12:09:47.979560 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567122 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 12:09:47.979560 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567125 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 12:09:47.979560 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567127 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 12:09:47.979560 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567130 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 12:09:47.979560 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567133 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 12:09:47.979560 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567135 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 12:09:47.979560 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567137 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 12:09:47.979560 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567140 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 12:09:47.979560 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567143 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 12:09:47.979560 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567145 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 12:09:47.979560 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567148 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 12:09:47.979560 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567150 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 12:09:47.979560 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567153 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 12:09:47.980122 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567156 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 12:09:47.980122 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567158 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 12:09:47.980122 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567161 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 12:09:47.980122 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567167 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 12:09:47.980122 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567171 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 12:09:47.980122 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567174 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 12:09:47.980122 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567177 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 12:09:47.980122 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567180 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 12:09:47.980122 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567183 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 12:09:47.980122 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567186 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 12:09:47.980122 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567188 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 12:09:47.980122 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.567194 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 12:09:47.980122 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567329 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 12:09:47.980122 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567338 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 12:09:47.980122 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567343 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 12:09:47.980594 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567347 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 12:09:47.980594 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567350 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 12:09:47.980594 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567353 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 12:09:47.980594 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567356 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 12:09:47.980594 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567359 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 12:09:47.980594 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567362 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 12:09:47.980594 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567364 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 12:09:47.980594 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567367 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 12:09:47.980594 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567369 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 12:09:47.980594 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567372 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 12:09:47.980594 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567375 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 12:09:47.980594 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567377 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 12:09:47.980594 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567380 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 12:09:47.980594 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567382 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 12:09:47.980594 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567385 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 12:09:47.980594 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567387 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 12:09:47.980594 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567390 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 12:09:47.980594 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567393 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 12:09:47.980594 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567395 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 12:09:47.980594 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567398 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 12:09:47.981145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567400 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 12:09:47.981145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567405 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 12:09:47.981145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567408 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 12:09:47.981145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567410 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 12:09:47.981145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567413 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 12:09:47.981145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567416 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 12:09:47.981145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567419 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 12:09:47.981145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567422 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 12:09:47.981145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567424 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 12:09:47.981145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567428 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 12:09:47.981145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567430 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 12:09:47.981145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567433 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 12:09:47.981145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567436 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 12:09:47.981145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567438 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 12:09:47.981145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567441 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 12:09:47.981145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567443 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 12:09:47.981145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567446 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 12:09:47.981145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567448 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 12:09:47.981145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567451 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 12:09:47.981145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567454 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 12:09:47.981712 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567456 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 12:09:47.981712 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567459 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 12:09:47.981712 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567463 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 12:09:47.981712 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567467 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 12:09:47.981712 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567470 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 12:09:47.981712 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567473 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 12:09:47.981712 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567475 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 12:09:47.981712 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567479 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 12:09:47.981712 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567482 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 12:09:47.981712 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567484 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 12:09:47.981712 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567487 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 12:09:47.981712 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567490 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 12:09:47.981712 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567493 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 12:09:47.981712 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567497 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 12:09:47.981712 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567500 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 12:09:47.981712 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567502 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 12:09:47.981712 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567505 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 12:09:47.981712 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567508 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 12:09:47.981712 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567511 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 12:09:47.982249 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567513 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 12:09:47.982249 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567516 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 12:09:47.982249 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567521 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 12:09:47.982249 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567523 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 12:09:47.982249 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567526 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 12:09:47.982249 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567528 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 12:09:47.982249 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567531 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 12:09:47.982249 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567533 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 12:09:47.982249 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567537 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 12:09:47.982249 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567540 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 12:09:47.982249 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567543 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 12:09:47.982249 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567545 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 12:09:47.982249 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567548 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 12:09:47.982249 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567550 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 12:09:47.982249 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567553 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 12:09:47.982249 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567555 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 12:09:47.982249 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567558 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 12:09:47.982249 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567563 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 12:09:47.982249 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567566 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 12:09:47.982767 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567569 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 12:09:47.982767 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567572 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 12:09:47.982767 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567575 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 12:09:47.982767 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567578 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 12:09:47.982767 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567580 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 12:09:47.982767 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567584 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 12:09:47.982767 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567587 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 12:09:47.982767 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567590 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 12:09:47.982767 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567593 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 12:09:47.982767 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567596 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 12:09:47.982767 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567599 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 12:09:47.982767 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567601 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 12:09:47.982767 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567604 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 12:09:47.982767 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567607 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 12:09:47.982767 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567610 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 12:09:47.982767 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567614 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 12:09:47.982767 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567617 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 12:09:47.982767 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:47.567619 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 12:09:47.983283 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.567624 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 12:09:47.983283 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.567759 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 23 12:09:47.983283 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.569937 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 23 12:09:47.983283 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.571371 2578 server.go:1019] "Starting client certificate rotation" Jul 23 12:09:47.983283 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.571481 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 12:09:47.983283 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.571521 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 12:09:47.983283 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.573894 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 12:09:47.983283 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.575128 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 12:09:47.983283 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.586749 2578 log.go:25] "Validated CRI v1 runtime API" Jul 23 12:09:47.983283 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.589268 2578 log.go:25] "Validated CRI v1 image API" Jul 23 12:09:47.983283 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.592620 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 23 12:09:47.983283 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.596057 2578 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 da3fbb07-b009-422d-9665-f276012410fd:/dev/nvme0n1p3 e7a21fe1-ca66-4884-9a9f-cd77dec252df:/dev/nvme0n1p4] Jul 23 12:09:47.983645 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.596073 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 23 12:09:47.983746 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.602399 2578 manager.go:216] Machine: {Timestamp:2026-07-23 12:09:47.601047081 +0000 UTC m=+0.363455184 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3115035 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2170ad1fac7aed502a89c68ccb429c SystemUUID:ec2170ad-1fac-7aed-502a-89c68ccb429c BootID:624b414f-2ceb-4c2f-b87d-7c7c319dcfa6 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b7:b5:b4:88:f5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b7:b5:b4:88:f5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:46:7e:7d:63:ab:93 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 23 12:09:47.983746 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.602506 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 23 12:09:47.983746 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.602595 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 23 12:09:47.983746 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.605682 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 23 12:09:47.983746 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.605708 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-37.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 23 12:09:47.983746 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.606496 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 23 12:09:47.983746 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.606507 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 23 12:09:47.983746 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.606515 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.607235 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.607288 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.608642 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.611284 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.611421 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.611445 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.611455 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.611469 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.611479 2578 kubelet.go:398] "Adding apiserver pod source" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.611489 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.611843 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.611864 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.614096 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.616058 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.616085 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.617196 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.617210 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.617216 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.617222 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.617230 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.617235 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.617244 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.617249 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.617255 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.617261 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.617270 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.617275 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.617960 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.617966 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.621305 2578 server.go:1268] "Started kubelet" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.621386 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.621426 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.621508 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.623007 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-37.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:47.623130 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:47.623216 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-37.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.624567 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.624960 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 23 12:09:47.984303 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:47.627481 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:47.630068 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-37.ec2.internal.18c4e9f4413b9d98 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-37.ec2.internal,UID:ip-10-0-138-37.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-37.ec2.internal,},FirstTimestamp:2026-07-23 12:09:47.62127708 +0000 UTC m=+0.383685184,LastTimestamp:2026-07-23 12:09:47.62127708 +0000 UTC m=+0.383685184,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-37.ec2.internal,}" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.631359 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.631364 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:47.631452 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.631920 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.631934 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.631966 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.632304 2578 factory.go:55] Registering systemd factory Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.632349 2578 factory.go:223] Registration of the systemd container factory successfully Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.632584 2578 factory.go:153] Registering CRI-O factory Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.632597 2578 factory.go:223] Registration of the crio container factory successfully Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.632927 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.632042 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.632951 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.632954 2578 factory.go:103] Registering Raw factory Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.632971 2578 manager.go:1196] Started watching for new ooms in manager Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.633926 2578 manager.go:349] Starting recovery of all containers Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:47.636782 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-37.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:47.641476 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.644892 2578 manager.go:354] Recovery completed Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.649961 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.652557 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientMemory" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.652603 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.652614 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientPID" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.653436 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.653449 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.653461 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.653480 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.653825 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-st5gs" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.654494 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientMemory" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.654536 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 12:09:47.985430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.654551 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientPID" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:47.655216 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-37.ec2.internal.18c4e9f4431952c5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-37.ec2.internal,UID:ip-10-0-138-37.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-37.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-37.ec2.internal,},FirstTimestamp:2026-07-23 12:09:47.652584133 +0000 UTC m=+0.414992238,LastTimestamp:2026-07-23 12:09:47.652584133 +0000 UTC m=+0.414992238,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-37.ec2.internal,}" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.655504 2578 policy_none.go:49] "None policy: Start" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.655519 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.655529 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.656625 2578 policy_none.go:47] "Start" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.661827 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-st5gs" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:47.663926 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-37.ec2.internal.18c4e9f44319b3ed default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-37.ec2.internal,UID:ip-10-0-138-37.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-37.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-37.ec2.internal,},FirstTimestamp:2026-07-23 12:09:47.652609005 +0000 UTC m=+0.415017107,LastTimestamp:2026-07-23 12:09:47.652609005 +0000 UTC m=+0.415017107,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-37.ec2.internal,}" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.713842 2578 manager.go:337] "Starting Device Plugin manager" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:47.713879 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.713889 2578 server.go:87] "Starting device plugin registration server" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.714128 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.714138 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.714217 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.714302 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.714310 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:47.714929 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:47.714971 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-37.ec2.internal\" not found" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.756975 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.758181 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.758219 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.758245 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:47.758282 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.760558 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.814614 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.815608 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientMemory" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.815632 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.815642 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientPID" Jul 23 12:09:48.132855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.815665 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-37.ec2.internal" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.826001 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-37.ec2.internal" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:47.826020 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-37.ec2.internal\": node \"ip-10-0-138-37.ec2.internal\" not found" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.858920 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal"] Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.858989 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.859840 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientMemory" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.859872 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.859888 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientPID" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.861072 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.861177 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.861203 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.862882 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientMemory" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.862906 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.862920 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientPID" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.862970 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientMemory" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.862996 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.863008 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientPID" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.864440 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.864463 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.865144 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientMemory" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.865171 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.865185 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientPID" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:47.895217 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-37.ec2.internal\" not found" node="ip-10-0-138-37.ec2.internal" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:47.898548 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-37.ec2.internal\" not found" node="ip-10-0-138-37.ec2.internal" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:47.906526 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.934344 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5fb9e113ed4aa68386e433953fb9a7e6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal\" (UID: \"5fb9e113ed4aa68386e433953fb9a7e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.934371 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fb9e113ed4aa68386e433953fb9a7e6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal\" (UID: \"5fb9e113ed4aa68386e433953fb9a7e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal" Jul 23 12:09:48.133912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:47.934387 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/542075cdbfd82dba7e896ceb37827ec4-config\") pod \"kube-apiserver-proxy-ip-10-0-138-37.ec2.internal\" (UID: \"542075cdbfd82dba7e896ceb37827ec4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal" Jul 23 12:09:48.134835 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:48.006756 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found" Jul 23 12:09:48.134835 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:48.034625 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5fb9e113ed4aa68386e433953fb9a7e6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal\" (UID: \"5fb9e113ed4aa68386e433953fb9a7e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal" Jul 23 12:09:48.134835 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:48.034650 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fb9e113ed4aa68386e433953fb9a7e6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal\" (UID: \"5fb9e113ed4aa68386e433953fb9a7e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal" Jul 23 12:09:48.134835 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:48.034667 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/542075cdbfd82dba7e896ceb37827ec4-config\") pod \"kube-apiserver-proxy-ip-10-0-138-37.ec2.internal\" (UID: \"542075cdbfd82dba7e896ceb37827ec4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal" Jul 23 12:09:48.134835 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:48.034705 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/542075cdbfd82dba7e896ceb37827ec4-config\") pod \"kube-apiserver-proxy-ip-10-0-138-37.ec2.internal\" (UID: \"542075cdbfd82dba7e896ceb37827ec4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal" Jul 23 12:09:48.134835 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:48.034741 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5fb9e113ed4aa68386e433953fb9a7e6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal\" (UID: \"5fb9e113ed4aa68386e433953fb9a7e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal" Jul 23 12:09:48.134835 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:48.034741 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fb9e113ed4aa68386e433953fb9a7e6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal\" (UID: \"5fb9e113ed4aa68386e433953fb9a7e6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal" Jul 23 12:09:48.134835 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:48.107603 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found" Jul 23 12:09:48.253610 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:48.198374 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal" Jul 23 12:09:48.253610 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:48.202364 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal" Jul 23 12:09:48.253610 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:48.208244 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found" Jul 23 12:09:48.371701 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:48.308737 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found" Jul 23 12:09:49.658394 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:48.409255 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found" Jul 23 12:09:49.773119 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:49.679537 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found" Jul 23 12:09:49.773119 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:49.741041 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 23 12:09:49.773119 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:49.741218 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 12:09:49.781224 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:49.780422 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found" Jul 23 12:09:49.899965 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:49.836307 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 12:04:48 +0000 UTC" deadline="2028-05-06 09:21:22.706447208 +0000 UTC" Jul 23 12:09:49.899965 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:49.836344 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15669h11m32.870107553s" Jul 23 12:09:49.899965 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:49.881430 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found" Jul 23 12:09:50.025237 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:49.918042 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 12:09:50.025237 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:49.919229 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientMemory" Jul 23 12:09:50.025237 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:49.919259 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 12:09:50.025237 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:49.919268 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeHasSufficientPID" Jul 23 12:09:50.025237 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:49.981661 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found" Jul 23 12:09:50.159201 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:50.082133 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found" Jul 23 12:09:50.285500 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:50.182297 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found" Jul 23 12:09:50.285500 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:50.282991 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found" Jul 23 12:09:50.397222 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:50.383585 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found" Jul 23 12:09:50.504378 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:50.484115 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-37.ec2.internal\" not found" Jul 23 12:09:50.554333 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.554308 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 12:09:50.556381 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.556364 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 12:09:50.558592 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.558572 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 12:09:50.602695 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.602671 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal" Jul 23 12:09:50.620503 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.620486 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 12:09:50.621996 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.621985 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal" Jul 23 12:09:50.633477 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.633461 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 12:09:50.731236 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.731212 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 12:09:50.783455 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.783393 2578 apiserver.go:52] "Watching apiserver" Jul 23 12:09:50.791651 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.791627 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 23 12:09:50.792002 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.791978 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal","openshift-network-diagnostics/network-check-target-tjdx4","openshift-network-operator/iptables-alerter-lzcnk","openshift-ovn-kubernetes/ovnkube-node-jsxrj","kube-system/global-pull-secret-syncer-tcgfq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js","openshift-cluster-node-tuning-operator/tuned-hwrnq","openshift-dns/node-resolver-rj4l2","openshift-image-registry/node-ca-zdc7h","openshift-multus/multus-additional-cni-plugins-sdx4n","openshift-multus/multus-fjtdt","openshift-multus/network-metrics-daemon-cbm6t","kube-system/konnectivity-agent-jl6fz","kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal"] Jul 23 12:09:50.794177 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.794158 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.796061 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.796039 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jl6fz" Jul 23 12:09:50.797137 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.796815 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jzs64\"" Jul 23 12:09:50.797137 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.797024 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 23 12:09:50.797287 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.797150 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lzcnk" Jul 23 12:09:50.797340 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.797289 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 23 12:09:50.798566 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.798549 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.799009 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.798990 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5v6nm\"" Jul 23 12:09:50.799102 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.799041 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 23 12:09:50.799457 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.799436 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 23 12:09:50.799540 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.799521 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 23 12:09:50.799540 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.799522 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 23 12:09:50.799704 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.799598 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zk7gx\"" Jul 23 12:09:50.799704 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.799608 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 23 12:09:50.800502 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.800482 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:09:50.800591 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:50.800569 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcgfq" podUID="88f19386-8325-4469-9859-cd876612dcd6" Jul 23 12:09:50.801805 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.801790 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 23 12:09:50.801937 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.801921 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.803804 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.803789 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zdc7h" Jul 23 12:09:50.805156 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.805142 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rj4l2" Jul 23 12:09:50.806268 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.806251 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:09:50.806361 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:50.806318 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjdx4" podUID="6b308275-101d-4a54-86d4-fc63739354cb" Jul 23 12:09:50.807684 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.807665 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:50.808431 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.808413 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sjpkj\"" Jul 23 12:09:50.808507 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.808443 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 23 12:09:50.808507 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.808455 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 23 12:09:50.808736 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.808720 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 23 12:09:50.809184 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.809164 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 23 12:09:50.809275 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.809209 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dphzz\"" Jul 23 12:09:50.809275 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.809239 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 23 12:09:50.809275 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.809256 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 23 12:09:50.809619 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.809602 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 23 12:09:50.809718 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.809623 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 23 12:09:50.809718 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.809641 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 23 12:09:50.809718 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.809650 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 23 12:09:50.809864 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.809797 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fjtdt" Jul 23 12:09:50.810879 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.810859 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 23 12:09:50.811179 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811159 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:09:50.811272 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:50.811247 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbm6t" podUID="232bfbf8-288e-4679-8bee-4c5179866c00" Jul 23 12:09:50.811380 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811362 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/024307b2-8735-4d0a-8673-0152c47b31bc-ovnkube-script-lib\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.811468 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811386 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/88f19386-8325-4469-9859-cd876612dcd6-dbus\") pod \"global-pull-secret-syncer-tcgfq\" (UID: \"88f19386-8325-4469-9859-cd876612dcd6\") " pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:09:50.811468 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811403 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.811468 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:50.811399 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod542075cdbfd82dba7e896ceb37827ec4.slice/crio-c44c5da73ac493c630be697451df21805a150667d2350e4b59f63f4a95b6004f WatchSource:0}: Error finding container c44c5da73ac493c630be697451df21805a150667d2350e4b59f63f4a95b6004f: Status 404 returned error can't find the container with id c44c5da73ac493c630be697451df21805a150667d2350e4b59f63f4a95b6004f Jul 23 12:09:50.811468 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811420 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3ee343ae-feea-4448-8ed4-641274b5be7e-agent-certs\") pod \"konnectivity-agent-jl6fz\" (UID: \"3ee343ae-feea-4448-8ed4-641274b5be7e\") " pod="kube-system/konnectivity-agent-jl6fz" Jul 23 12:09:50.811468 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811438 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-systemd-units\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.811468 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811461 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-run-netns\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.811736 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811483 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h72vd\" (UniqueName: \"kubernetes.io/projected/024307b2-8735-4d0a-8673-0152c47b31bc-kube-api-access-h72vd\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.811736 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811532 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-67r85\"" Jul 23 12:09:50.811736 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811544 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-device-dir\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.811736 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811567 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-etc-selinux\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.811736 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811589 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-sys-fs\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.811736 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811612 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-sysctl-conf\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.811736 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811645 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-tuned\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.811736 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811681 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z84hk\" (UniqueName: \"kubernetes.io/projected/26f8cdf9-5ba7-4629-95a1-49f140dcecde-kube-api-access-z84hk\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.811736 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811698 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/024307b2-8735-4d0a-8673-0152c47b31bc-ovnkube-config\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.811736 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:50.811699 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fb9e113ed4aa68386e433953fb9a7e6.slice/crio-e4be1891498863c520b99b334f11bde740e334d9e9290a6eaaf40025c32c6be0 WatchSource:0}: Error finding container e4be1891498863c520b99b334f11bde740e334d9e9290a6eaaf40025c32c6be0: Status 404 returned error can't find the container with id e4be1891498863c520b99b334f11bde740e334d9e9290a6eaaf40025c32c6be0 Jul 23 12:09:50.811736 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811712 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/024307b2-8735-4d0a-8673-0152c47b31bc-ovn-node-metrics-cert\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.811736 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811732 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cbede117-0113-4f86-a8b7-532164bf587f-host-slash\") pod \"iptables-alerter-lzcnk\" (UID: \"cbede117-0113-4f86-a8b7-532164bf587f\") " pod="openshift-network-operator/iptables-alerter-lzcnk" Jul 23 12:09:50.812273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811754 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlx2v\" (UniqueName: \"kubernetes.io/projected/cbede117-0113-4f86-a8b7-532164bf587f-kube-api-access-jlx2v\") pod \"iptables-alerter-lzcnk\" (UID: \"cbede117-0113-4f86-a8b7-532164bf587f\") " pod="openshift-network-operator/iptables-alerter-lzcnk" Jul 23 12:09:50.812273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811777 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26f8cdf9-5ba7-4629-95a1-49f140dcecde-tmp\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.812273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811806 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 23 12:09:50.812273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811839 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 23 12:09:50.812273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811890 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 23 12:09:50.812273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811809 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-slash\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.812273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811782 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 23 12:09:50.812273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811927 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 23 12:09:50.812273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.811962 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-var-lib-openvswitch\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.812273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812048 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-run-ovn-kubernetes\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.812273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812064 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-cni-netd\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.812273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812084 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-socket-dir\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.812273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812096 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 23 12:09:50.812273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812105 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/374f6cfe-ca3e-424f-9875-3040a3e78447-host\") pod \"node-ca-zdc7h\" (UID: \"374f6cfe-ca3e-424f-9875-3040a3e78447\") " pod="openshift-image-registry/node-ca-zdc7h" Jul 23 12:09:50.812273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812141 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-run-ovn\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.812273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812156 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-var-lib-kubelet\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.812273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812223 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-host\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.812985 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812242 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-kubernetes\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.812985 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812254 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-sys\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.812985 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812280 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3ee343ae-feea-4448-8ed4-641274b5be7e-konnectivity-ca\") pod \"konnectivity-agent-jl6fz\" (UID: \"3ee343ae-feea-4448-8ed4-641274b5be7e\") " pod="kube-system/konnectivity-agent-jl6fz" Jul 23 12:09:50.812985 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812299 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-node-log\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.812985 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812327 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-cni-bin\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.812985 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812343 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-systemd\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.812985 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812378 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-run\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.812985 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812397 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-kubelet\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.812985 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812410 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-etc-openvswitch\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.812985 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812429 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/024307b2-8735-4d0a-8673-0152c47b31bc-env-overrides\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.812985 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812456 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-registration-dir\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.812985 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812470 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-sysctl-d\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.812985 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812483 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-log-socket\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.812985 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812499 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-run-openvswitch\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.813601 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812533 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.813601 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812553 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cbede117-0113-4f86-a8b7-532164bf587f-iptables-alerter-script\") pod \"iptables-alerter-lzcnk\" (UID: \"cbede117-0113-4f86-a8b7-532164bf587f\") " pod="openshift-network-operator/iptables-alerter-lzcnk" Jul 23 12:09:50.813601 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812569 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/88f19386-8325-4469-9859-cd876612dcd6-kubelet-config\") pod \"global-pull-secret-syncer-tcgfq\" (UID: \"88f19386-8325-4469-9859-cd876612dcd6\") " pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:09:50.813601 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812587 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret\") pod \"global-pull-secret-syncer-tcgfq\" (UID: \"88f19386-8325-4469-9859-cd876612dcd6\") " pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:09:50.813601 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812608 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rp42\" (UniqueName: \"kubernetes.io/projected/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-kube-api-access-8rp42\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.813601 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812623 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/374f6cfe-ca3e-424f-9875-3040a3e78447-serviceca\") pod \"node-ca-zdc7h\" (UID: \"374f6cfe-ca3e-424f-9875-3040a3e78447\") " pod="openshift-image-registry/node-ca-zdc7h" Jul 23 12:09:50.813601 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812636 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd4mc\" (UniqueName: \"kubernetes.io/projected/374f6cfe-ca3e-424f-9875-3040a3e78447-kube-api-access-sd4mc\") pod \"node-ca-zdc7h\" (UID: \"374f6cfe-ca3e-424f-9875-3040a3e78447\") " pod="openshift-image-registry/node-ca-zdc7h" Jul 23 12:09:50.813601 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812650 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-run-systemd\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.813601 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812671 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-lib-modules\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.813601 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.812726 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-modprobe-d\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.813601 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.813230 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-sysconfig\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.814960 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.814939 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5gvz5\"" Jul 23 12:09:50.814960 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.814953 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 23 12:09:50.815404 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.815382 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 23 12:09:50.815963 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.815920 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bnwb8\"" Jul 23 12:09:50.818214 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.816467 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 23 12:09:50.818214 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.817187 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-sqjlk\"" Jul 23 12:09:50.818618 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.818251 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 12:09:50.822905 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.822889 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 12:09:50.847295 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.847277 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kfqtv" Jul 23 12:09:50.857999 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.857982 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kfqtv" Jul 23 12:09:50.903519 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.903488 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 23 12:09:50.914073 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914052 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-cni-netd\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.914144 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914081 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-system-cni-dir\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:50.914144 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914132 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-host-run-k8s-cni-cncf-io\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:50.914225 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914169 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-cni-netd\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.914225 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914172 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-kubernetes\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.914291 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914230 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-node-log\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.914291 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914254 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-cni-bin\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.914291 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914271 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-multus-cni-dir\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:50.914291 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914232 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-kubernetes\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.914401 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914327 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-node-log\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.914401 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914286 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-system-cni-dir\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:50.914401 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914354 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-cnibin\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:50.914401 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914389 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-cni-bin\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.914401 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914393 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs\") pod \"network-metrics-daemon-cbm6t\" (UID: \"232bfbf8-288e-4679-8bee-4c5179866c00\") " pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:09:50.914541 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914416 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-run\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.914541 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914431 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-kubelet\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.914541 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914446 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-etc-openvswitch\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.914541 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914487 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-kubelet\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.914541 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914496 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-registration-dir\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.914541 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914514 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-cnibin\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:50.914541 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914528 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-registration-dir\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.914541 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914529 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-os-release\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:50.914541 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914487 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-etc-openvswitch\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.914793 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914549 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-log-socket\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.914793 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914549 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-run\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.914793 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914582 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-log-socket\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.914793 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914606 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/055376f5-9608-4f55-b8ae-e6f8b9d2329e-cni-binary-copy\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:50.914793 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914623 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-host-var-lib-cni-bin\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:50.914793 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914639 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:50.914793 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914657 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sqkm\" (UniqueName: \"kubernetes.io/projected/232bfbf8-288e-4679-8bee-4c5179866c00-kube-api-access-8sqkm\") pod \"network-metrics-daemon-cbm6t\" (UID: \"232bfbf8-288e-4679-8bee-4c5179866c00\") " pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:09:50.914793 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914672 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2rbr\" (UniqueName: \"kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr\") pod \"network-check-target-tjdx4\" (UID: \"6b308275-101d-4a54-86d4-fc63739354cb\") " pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:09:50.914793 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914699 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cbede117-0113-4f86-a8b7-532164bf587f-iptables-alerter-script\") pod \"iptables-alerter-lzcnk\" (UID: \"cbede117-0113-4f86-a8b7-532164bf587f\") " pod="openshift-network-operator/iptables-alerter-lzcnk" Jul 23 12:09:50.914793 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914717 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/88f19386-8325-4469-9859-cd876612dcd6-kubelet-config\") pod \"global-pull-secret-syncer-tcgfq\" (UID: \"88f19386-8325-4469-9859-cd876612dcd6\") " pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:09:50.914793 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914733 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret\") pod \"global-pull-secret-syncer-tcgfq\" (UID: \"88f19386-8325-4469-9859-cd876612dcd6\") " pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:09:50.914793 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914748 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rp42\" (UniqueName: \"kubernetes.io/projected/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-kube-api-access-8rp42\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.914793 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914763 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-run-systemd\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.914793 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914784 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/024307b2-8735-4d0a-8673-0152c47b31bc-ovnkube-script-lib\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.915432 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914807 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/88f19386-8325-4469-9859-cd876612dcd6-dbus\") pod \"global-pull-secret-syncer-tcgfq\" (UID: \"88f19386-8325-4469-9859-cd876612dcd6\") " pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:09:50.915432 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914812 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/88f19386-8325-4469-9859-cd876612dcd6-kubelet-config\") pod \"global-pull-secret-syncer-tcgfq\" (UID: \"88f19386-8325-4469-9859-cd876612dcd6\") " pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:09:50.915432 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:50.914841 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 12:09:50.915432 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914864 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-run-systemd\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.915432 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914877 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.915432 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914845 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.915432 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914938 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/88f19386-8325-4469-9859-cd876612dcd6-dbus\") pod \"global-pull-secret-syncer-tcgfq\" (UID: \"88f19386-8325-4469-9859-cd876612dcd6\") " pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:09:50.915432 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:50.914942 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret podName:88f19386-8325-4469-9859-cd876612dcd6 nodeName:}" failed. No retries permitted until 2026-07-23 12:09:51.414884531 +0000 UTC m=+3.007419208 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret") pod "global-pull-secret-syncer-tcgfq" (UID: "88f19386-8325-4469-9859-cd876612dcd6") : object "kube-system"/"original-pull-secret" not registered Jul 23 12:09:50.915432 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914979 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-sys-fs\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.915432 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.914997 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-run-netns\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.915432 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915011 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h72vd\" (UniqueName: \"kubernetes.io/projected/024307b2-8735-4d0a-8673-0152c47b31bc-kube-api-access-h72vd\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.915432 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915029 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-device-dir\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.915432 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915055 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:50.915432 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915063 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-sys-fs\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.916102 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915066 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-run-netns\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.916102 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915080 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-sysctl-conf\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.916102 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915110 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-tuned\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.916102 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915095 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-device-dir\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.916102 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915134 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z84hk\" (UniqueName: \"kubernetes.io/projected/26f8cdf9-5ba7-4629-95a1-49f140dcecde-kube-api-access-z84hk\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.916102 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915156 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/024307b2-8735-4d0a-8673-0152c47b31bc-ovnkube-config\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.916102 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915180 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cbede117-0113-4f86-a8b7-532164bf587f-host-slash\") pod \"iptables-alerter-lzcnk\" (UID: \"cbede117-0113-4f86-a8b7-532164bf587f\") " pod="openshift-network-operator/iptables-alerter-lzcnk" Jul 23 12:09:50.916102 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915220 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-sysctl-conf\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.916102 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915229 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlx2v\" (UniqueName: \"kubernetes.io/projected/cbede117-0113-4f86-a8b7-532164bf587f-kube-api-access-jlx2v\") pod \"iptables-alerter-lzcnk\" (UID: \"cbede117-0113-4f86-a8b7-532164bf587f\") " pod="openshift-network-operator/iptables-alerter-lzcnk" Jul 23 12:09:50.916102 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915270 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cbede117-0113-4f86-a8b7-532164bf587f-host-slash\") pod \"iptables-alerter-lzcnk\" (UID: \"cbede117-0113-4f86-a8b7-532164bf587f\") " pod="openshift-network-operator/iptables-alerter-lzcnk" Jul 23 12:09:50.916102 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915296 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-socket-dir\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.916102 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915320 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cbede117-0113-4f86-a8b7-532164bf587f-iptables-alerter-script\") pod \"iptables-alerter-lzcnk\" (UID: \"cbede117-0113-4f86-a8b7-532164bf587f\") " pod="openshift-network-operator/iptables-alerter-lzcnk" Jul 23 12:09:50.916102 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915320 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-host-var-lib-cni-multus\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:50.916102 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915370 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26f8cdf9-5ba7-4629-95a1-49f140dcecde-tmp\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.916678 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915393 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-slash\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.916678 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915440 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-var-lib-openvswitch\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.916678 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915455 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/024307b2-8735-4d0a-8673-0152c47b31bc-ovnkube-script-lib\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.916678 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915464 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-run-ovn-kubernetes\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.916678 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915456 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-socket-dir\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.916678 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915488 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/374f6cfe-ca3e-424f-9875-3040a3e78447-host\") pod \"node-ca-zdc7h\" (UID: \"374f6cfe-ca3e-424f-9875-3040a3e78447\") " pod="openshift-image-registry/node-ca-zdc7h" Jul 23 12:09:50.916678 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915494 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-var-lib-openvswitch\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.916678 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915515 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-run-ovn-kubernetes\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.916678 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915480 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-slash\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.916678 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915517 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-host-var-lib-kubelet\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:50.916678 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915534 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/374f6cfe-ca3e-424f-9875-3040a3e78447-host\") pod \"node-ca-zdc7h\" (UID: \"374f6cfe-ca3e-424f-9875-3040a3e78447\") " pod="openshift-image-registry/node-ca-zdc7h" Jul 23 12:09:50.916678 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915571 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/055376f5-9608-4f55-b8ae-e6f8b9d2329e-multus-daemon-config\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:50.916678 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915596 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 23 12:09:50.916678 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915616 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcxkh\" (UniqueName: \"kubernetes.io/projected/d3c0c881-554f-4572-bdaa-c0ff060c50ec-kube-api-access-tcxkh\") pod \"node-resolver-rj4l2\" (UID: \"d3c0c881-554f-4572-bdaa-c0ff060c50ec\") " pod="openshift-dns/node-resolver-rj4l2" Jul 23 12:09:50.916678 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915652 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-run-ovn\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.917409 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915676 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-var-lib-kubelet\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.917409 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915695 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-host\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.917409 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915715 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-sys\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.917409 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915721 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/024307b2-8735-4d0a-8673-0152c47b31bc-ovnkube-config\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.917409 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915735 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3ee343ae-feea-4448-8ed4-641274b5be7e-konnectivity-ca\") pod \"konnectivity-agent-jl6fz\" (UID: \"3ee343ae-feea-4448-8ed4-641274b5be7e\") " pod="kube-system/konnectivity-agent-jl6fz" Jul 23 12:09:50.917409 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915769 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-var-lib-kubelet\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.917409 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915775 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-etc-kubernetes\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:50.917409 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915828 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d3c0c881-554f-4572-bdaa-c0ff060c50ec-hosts-file\") pod \"node-resolver-rj4l2\" (UID: \"d3c0c881-554f-4572-bdaa-c0ff060c50ec\") " pod="openshift-dns/node-resolver-rj4l2" Jul 23 12:09:50.917409 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915834 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-run-ovn\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.917409 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915854 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-systemd\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.917409 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915879 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/024307b2-8735-4d0a-8673-0152c47b31bc-env-overrides\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.917409 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915904 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-multus-socket-dir-parent\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:50.917409 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915928 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-host-run-netns\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:50.917409 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915941 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-host\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.917409 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916005 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-systemd\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.918136 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.915951 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4v8l\" (UniqueName: \"kubernetes.io/projected/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-kube-api-access-t4v8l\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:50.918136 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916026 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-sys\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.918136 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916120 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-sysctl-d\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.918136 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916146 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn87w\" (UniqueName: \"kubernetes.io/projected/055376f5-9608-4f55-b8ae-e6f8b9d2329e-kube-api-access-zn87w\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:50.918136 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916170 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:50.918136 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916212 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-run-openvswitch\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.918136 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916235 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.918136 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916258 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd4mc\" (UniqueName: \"kubernetes.io/projected/374f6cfe-ca3e-424f-9875-3040a3e78447-kube-api-access-sd4mc\") pod \"node-ca-zdc7h\" (UID: \"374f6cfe-ca3e-424f-9875-3040a3e78447\") " pod="openshift-image-registry/node-ca-zdc7h" Jul 23 12:09:50.918136 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916264 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-sysctl-d\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.918136 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916277 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-lib-modules\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.918136 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916299 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-modprobe-d\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.918136 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916318 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-sysconfig\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.918136 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916277 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3ee343ae-feea-4448-8ed4-641274b5be7e-konnectivity-ca\") pod \"konnectivity-agent-jl6fz\" (UID: \"3ee343ae-feea-4448-8ed4-641274b5be7e\") " pod="kube-system/konnectivity-agent-jl6fz" Jul 23 12:09:50.918136 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916328 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.918808 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916338 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-etc-selinux\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.918808 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916360 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-os-release\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:50.918808 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916406 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-sysconfig\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.918808 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916409 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/024307b2-8735-4d0a-8673-0152c47b31bc-env-overrides\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.918808 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916413 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-hostroot\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:50.918808 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916317 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-run-openvswitch\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.918808 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916449 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-host-run-multus-certs\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:50.918808 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916504 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-lib-modules\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.918808 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916512 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-etc-selinux\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.918808 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916508 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-modprobe-d\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.918808 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916501 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/374f6cfe-ca3e-424f-9875-3040a3e78447-serviceca\") pod \"node-ca-zdc7h\" (UID: \"374f6cfe-ca3e-424f-9875-3040a3e78447\") " pod="openshift-image-registry/node-ca-zdc7h" Jul 23 12:09:50.918808 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916553 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3ee343ae-feea-4448-8ed4-641274b5be7e-agent-certs\") pod \"konnectivity-agent-jl6fz\" (UID: \"3ee343ae-feea-4448-8ed4-641274b5be7e\") " pod="kube-system/konnectivity-agent-jl6fz" Jul 23 12:09:50.918808 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916573 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-systemd-units\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.918808 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916600 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:50.919355 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916622 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/024307b2-8735-4d0a-8673-0152c47b31bc-systemd-units\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.919355 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916621 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3c0c881-554f-4572-bdaa-c0ff060c50ec-tmp-dir\") pod \"node-resolver-rj4l2\" (UID: \"d3c0c881-554f-4572-bdaa-c0ff060c50ec\") " pod="openshift-dns/node-resolver-rj4l2" Jul 23 12:09:50.919355 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916673 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/024307b2-8735-4d0a-8673-0152c47b31bc-ovn-node-metrics-cert\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.919355 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916737 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-multus-conf-dir\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:50.919355 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.916908 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/374f6cfe-ca3e-424f-9875-3040a3e78447-serviceca\") pod \"node-ca-zdc7h\" (UID: \"374f6cfe-ca3e-424f-9875-3040a3e78447\") " pod="openshift-image-registry/node-ca-zdc7h" Jul 23 12:09:50.919355 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.919064 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/26f8cdf9-5ba7-4629-95a1-49f140dcecde-etc-tuned\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.919355 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.919106 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/024307b2-8735-4d0a-8673-0152c47b31bc-ovn-node-metrics-cert\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.919355 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.919135 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26f8cdf9-5ba7-4629-95a1-49f140dcecde-tmp\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.919355 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.919217 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3ee343ae-feea-4448-8ed4-641274b5be7e-agent-certs\") pod \"konnectivity-agent-jl6fz\" (UID: \"3ee343ae-feea-4448-8ed4-641274b5be7e\") " pod="kube-system/konnectivity-agent-jl6fz" Jul 23 12:09:50.926937 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.926904 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z84hk\" (UniqueName: \"kubernetes.io/projected/26f8cdf9-5ba7-4629-95a1-49f140dcecde-kube-api-access-z84hk\") pod \"tuned-hwrnq\" (UID: \"26f8cdf9-5ba7-4629-95a1-49f140dcecde\") " pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:50.927328 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.927301 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd4mc\" (UniqueName: \"kubernetes.io/projected/374f6cfe-ca3e-424f-9875-3040a3e78447-kube-api-access-sd4mc\") pod \"node-ca-zdc7h\" (UID: \"374f6cfe-ca3e-424f-9875-3040a3e78447\") " pod="openshift-image-registry/node-ca-zdc7h" Jul 23 12:09:50.927328 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.927305 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlx2v\" (UniqueName: \"kubernetes.io/projected/cbede117-0113-4f86-a8b7-532164bf587f-kube-api-access-jlx2v\") pod \"iptables-alerter-lzcnk\" (UID: \"cbede117-0113-4f86-a8b7-532164bf587f\") " pod="openshift-network-operator/iptables-alerter-lzcnk" Jul 23 12:09:50.927501 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.927475 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rp42\" (UniqueName: \"kubernetes.io/projected/f4bf1809-2e30-4f4e-8e1d-059a5d44bec6-kube-api-access-8rp42\") pod \"aws-ebs-csi-driver-node-p55js\" (UID: \"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:50.927957 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.927937 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h72vd\" (UniqueName: \"kubernetes.io/projected/024307b2-8735-4d0a-8673-0152c47b31bc-kube-api-access-h72vd\") pod \"ovnkube-node-jsxrj\" (UID: \"024307b2-8735-4d0a-8673-0152c47b31bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:50.934952 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.934913 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal" event={"ID":"5fb9e113ed4aa68386e433953fb9a7e6","Type":"ContainerStarted","Data":"e4be1891498863c520b99b334f11bde740e334d9e9290a6eaaf40025c32c6be0"} Jul 23 12:09:50.935923 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:50.935905 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal" event={"ID":"542075cdbfd82dba7e896ceb37827ec4","Type":"ContainerStarted","Data":"c44c5da73ac493c630be697451df21805a150667d2350e4b59f63f4a95b6004f"} Jul 23 12:09:51.017940 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.017902 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-host-var-lib-kubelet\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.017940 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.017934 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/055376f5-9608-4f55-b8ae-e6f8b9d2329e-multus-daemon-config\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.018183 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.017952 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcxkh\" (UniqueName: \"kubernetes.io/projected/d3c0c881-554f-4572-bdaa-c0ff060c50ec-kube-api-access-tcxkh\") pod \"node-resolver-rj4l2\" (UID: \"d3c0c881-554f-4572-bdaa-c0ff060c50ec\") " pod="openshift-dns/node-resolver-rj4l2" Jul 23 12:09:51.018183 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.017969 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-etc-kubernetes\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.018183 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.017990 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d3c0c881-554f-4572-bdaa-c0ff060c50ec-hosts-file\") pod \"node-resolver-rj4l2\" (UID: \"d3c0c881-554f-4572-bdaa-c0ff060c50ec\") " pod="openshift-dns/node-resolver-rj4l2" Jul 23 12:09:51.018183 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018007 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-host-var-lib-kubelet\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.018183 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018055 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-multus-socket-dir-parent\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.018183 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018058 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d3c0c881-554f-4572-bdaa-c0ff060c50ec-hosts-file\") pod \"node-resolver-rj4l2\" (UID: \"d3c0c881-554f-4572-bdaa-c0ff060c50ec\") " pod="openshift-dns/node-resolver-rj4l2" Jul 23 12:09:51.018183 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018098 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-multus-socket-dir-parent\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.018183 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018087 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-host-run-netns\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.018183 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018129 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4v8l\" (UniqueName: \"kubernetes.io/projected/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-kube-api-access-t4v8l\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:51.018183 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018058 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-etc-kubernetes\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.018183 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018142 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-host-run-netns\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.018183 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018146 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn87w\" (UniqueName: \"kubernetes.io/projected/055376f5-9608-4f55-b8ae-e6f8b9d2329e-kube-api-access-zn87w\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.018183 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018187 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:51.018801 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018234 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-os-release\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.018801 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018250 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-hostroot\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.018801 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018272 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-host-run-multus-certs\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.018801 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018301 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:51.018801 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018311 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-os-release\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.018801 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018320 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-hostroot\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.018801 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018327 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3c0c881-554f-4572-bdaa-c0ff060c50ec-tmp-dir\") pod \"node-resolver-rj4l2\" (UID: \"d3c0c881-554f-4572-bdaa-c0ff060c50ec\") " pod="openshift-dns/node-resolver-rj4l2" Jul 23 12:09:51.018801 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018360 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-host-run-multus-certs\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.018801 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018373 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:51.018801 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018378 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-multus-conf-dir\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.018801 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018410 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-multus-conf-dir\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.018801 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018422 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-system-cni-dir\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.018801 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018445 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-host-run-k8s-cni-cncf-io\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.018801 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018470 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-multus-cni-dir\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.018801 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018493 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-system-cni-dir\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:51.019350 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018505 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-host-run-k8s-cni-cncf-io\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.019350 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018508 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-system-cni-dir\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.019350 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018521 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-cnibin\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:51.019350 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018545 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs\") pod \"network-metrics-daemon-cbm6t\" (UID: \"232bfbf8-288e-4679-8bee-4c5179866c00\") " pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:09:51.019350 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018549 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-system-cni-dir\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:51.019350 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018549 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-multus-cni-dir\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.019350 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018572 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-cnibin\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.019350 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018597 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-os-release\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:51.019350 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:51.018620 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 12:09:51.019350 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018647 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3c0c881-554f-4572-bdaa-c0ff060c50ec-tmp-dir\") pod \"node-resolver-rj4l2\" (UID: \"d3c0c881-554f-4572-bdaa-c0ff060c50ec\") " pod="openshift-dns/node-resolver-rj4l2" Jul 23 12:09:51.019350 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018620 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/055376f5-9608-4f55-b8ae-e6f8b9d2329e-cni-binary-copy\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.019350 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:51.018674 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs podName:232bfbf8-288e-4679-8bee-4c5179866c00 nodeName:}" failed. No retries permitted until 2026-07-23 12:09:51.518657262 +0000 UTC m=+3.111191939 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs") pod "network-metrics-daemon-cbm6t" (UID: "232bfbf8-288e-4679-8bee-4c5179866c00") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 12:09:51.019350 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018673 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-os-release\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:51.019350 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018558 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-cnibin\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:51.019740 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018668 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/055376f5-9608-4f55-b8ae-e6f8b9d2329e-multus-daemon-config\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.019740 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018710 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-host-var-lib-cni-bin\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.019740 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018680 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-cnibin\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.019740 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018728 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:51.019740 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018768 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sqkm\" (UniqueName: \"kubernetes.io/projected/232bfbf8-288e-4679-8bee-4c5179866c00-kube-api-access-8sqkm\") pod \"network-metrics-daemon-cbm6t\" (UID: \"232bfbf8-288e-4679-8bee-4c5179866c00\") " pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:09:51.019740 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018775 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-host-var-lib-cni-bin\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.019740 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018889 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2rbr\" (UniqueName: \"kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr\") pod \"network-check-target-tjdx4\" (UID: \"6b308275-101d-4a54-86d4-fc63739354cb\") " pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:09:51.019740 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018943 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:51.019740 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.018977 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-host-var-lib-cni-multus\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.019740 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.019046 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/055376f5-9608-4f55-b8ae-e6f8b9d2329e-cni-binary-copy\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.019740 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.019057 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/055376f5-9608-4f55-b8ae-e6f8b9d2329e-host-var-lib-cni-multus\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.019740 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.019259 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:51.019740 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.019354 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:51.019740 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.019534 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:51.040134 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.040075 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn87w\" (UniqueName: \"kubernetes.io/projected/055376f5-9608-4f55-b8ae-e6f8b9d2329e-kube-api-access-zn87w\") pod \"multus-fjtdt\" (UID: \"055376f5-9608-4f55-b8ae-e6f8b9d2329e\") " pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.042160 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.042134 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4v8l\" (UniqueName: \"kubernetes.io/projected/52f44df7-1c9e-4e78-9c0a-b989bf1f70c7-kube-api-access-t4v8l\") pod \"multus-additional-cni-plugins-sdx4n\" (UID: \"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7\") " pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:51.042734 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.042715 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcxkh\" (UniqueName: \"kubernetes.io/projected/d3c0c881-554f-4572-bdaa-c0ff060c50ec-kube-api-access-tcxkh\") pod \"node-resolver-rj4l2\" (UID: \"d3c0c881-554f-4572-bdaa-c0ff060c50ec\") " pod="openshift-dns/node-resolver-rj4l2" Jul 23 12:09:51.043581 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:51.043564 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 12:09:51.043665 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:51.043587 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 12:09:51.043665 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:51.043600 2578 projected.go:196] Error preparing data for projected volume kube-api-access-m2rbr for pod openshift-network-diagnostics/network-check-target-tjdx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 12:09:51.043771 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:51.043666 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr podName:6b308275-101d-4a54-86d4-fc63739354cb nodeName:}" failed. No retries permitted until 2026-07-23 12:09:51.54364878 +0000 UTC m=+3.136183467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-m2rbr" (UniqueName: "kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr") pod "network-check-target-tjdx4" (UID: "6b308275-101d-4a54-86d4-fc63739354cb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 12:09:51.047486 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.047466 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sqkm\" (UniqueName: \"kubernetes.io/projected/232bfbf8-288e-4679-8bee-4c5179866c00-kube-api-access-8sqkm\") pod \"network-metrics-daemon-cbm6t\" (UID: \"232bfbf8-288e-4679-8bee-4c5179866c00\") " pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:09:51.119045 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.119019 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" Jul 23 12:09:51.124869 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.124852 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jl6fz" Jul 23 12:09:51.127309 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:51.127280 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26f8cdf9_5ba7_4629_95a1_49f140dcecde.slice/crio-928407e07db5e17ffac56589b6cd1c597ed317c0439c96d6ca2e7e38e7127aa2 WatchSource:0}: Error finding container 928407e07db5e17ffac56589b6cd1c597ed317c0439c96d6ca2e7e38e7127aa2: Status 404 returned error can't find the container with id 928407e07db5e17ffac56589b6cd1c597ed317c0439c96d6ca2e7e38e7127aa2 Jul 23 12:09:51.132128 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.132111 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lzcnk" Jul 23 12:09:51.132832 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:51.132807 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ee343ae_feea_4448_8ed4_641274b5be7e.slice/crio-7cfd1efb4cc6645000ff3cb02da3ab2a999de1d29bc9189b909b08b74eaba26a WatchSource:0}: Error finding container 7cfd1efb4cc6645000ff3cb02da3ab2a999de1d29bc9189b909b08b74eaba26a: Status 404 returned error can't find the container with id 7cfd1efb4cc6645000ff3cb02da3ab2a999de1d29bc9189b909b08b74eaba26a Jul 23 12:09:51.137394 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.137373 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:09:51.139932 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:51.139909 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbede117_0113_4f86_a8b7_532164bf587f.slice/crio-a617f4441ebfa22809766f46bb987a34ceb2f774a2224b896fd6d2bf215d1d21 WatchSource:0}: Error finding container a617f4441ebfa22809766f46bb987a34ceb2f774a2224b896fd6d2bf215d1d21: Status 404 returned error can't find the container with id a617f4441ebfa22809766f46bb987a34ceb2f774a2224b896fd6d2bf215d1d21 Jul 23 12:09:51.142332 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.142287 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" Jul 23 12:09:51.147418 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.147397 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zdc7h" Jul 23 12:09:51.147753 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:51.147733 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod024307b2_8735_4d0a_8673_0152c47b31bc.slice/crio-a17ef3552b67bf398060131b2bcad9a78b5a32f26863b132766e4737e755fb33 WatchSource:0}: Error finding container a17ef3552b67bf398060131b2bcad9a78b5a32f26863b132766e4737e755fb33: Status 404 returned error can't find the container with id a17ef3552b67bf398060131b2bcad9a78b5a32f26863b132766e4737e755fb33 Jul 23 12:09:51.150702 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:51.150681 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4bf1809_2e30_4f4e_8e1d_059a5d44bec6.slice/crio-f8ed430fff2eb9a190fb019ad91782ad8421f4a0e601e80224e7123027e5bfe3 WatchSource:0}: Error finding container f8ed430fff2eb9a190fb019ad91782ad8421f4a0e601e80224e7123027e5bfe3: Status 404 returned error can't find the container with id f8ed430fff2eb9a190fb019ad91782ad8421f4a0e601e80224e7123027e5bfe3 Jul 23 12:09:51.151758 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.151739 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rj4l2" Jul 23 12:09:51.155132 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:51.155106 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod374f6cfe_ca3e_424f_9875_3040a3e78447.slice/crio-fbd77a930ee093d7bd0599c4167b0fef7da03505ba62930ad7a36708c15f4a9a WatchSource:0}: Error finding container fbd77a930ee093d7bd0599c4167b0fef7da03505ba62930ad7a36708c15f4a9a: Status 404 returned error can't find the container with id fbd77a930ee093d7bd0599c4167b0fef7da03505ba62930ad7a36708c15f4a9a Jul 23 12:09:51.156849 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.156828 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sdx4n" Jul 23 12:09:51.160228 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:51.160204 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3c0c881_554f_4572_bdaa_c0ff060c50ec.slice/crio-df7974e721bac0fa7d88d3db29522d1e0d8a156099b200cc3ba81fca8add7b11 WatchSource:0}: Error finding container df7974e721bac0fa7d88d3db29522d1e0d8a156099b200cc3ba81fca8add7b11: Status 404 returned error can't find the container with id df7974e721bac0fa7d88d3db29522d1e0d8a156099b200cc3ba81fca8add7b11 Jul 23 12:09:51.161976 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.161959 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fjtdt" Jul 23 12:09:51.165281 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:51.165248 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52f44df7_1c9e_4e78_9c0a_b989bf1f70c7.slice/crio-1a36ce2ef2596b7fce1b56dd34e5d8d1f893cbbfde31d48042011b5a7c35373f WatchSource:0}: Error finding container 1a36ce2ef2596b7fce1b56dd34e5d8d1f893cbbfde31d48042011b5a7c35373f: Status 404 returned error can't find the container with id 1a36ce2ef2596b7fce1b56dd34e5d8d1f893cbbfde31d48042011b5a7c35373f Jul 23 12:09:51.170001 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:09:51.169982 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod055376f5_9608_4f55_b8ae_e6f8b9d2329e.slice/crio-16e7229d3f4b8f34083f1cabe0377bed7c24bb4f2fd92c8e6ac98e6a41d4e686 WatchSource:0}: Error finding container 16e7229d3f4b8f34083f1cabe0377bed7c24bb4f2fd92c8e6ac98e6a41d4e686: Status 404 returned error can't find the container with id 16e7229d3f4b8f34083f1cabe0377bed7c24bb4f2fd92c8e6ac98e6a41d4e686 Jul 23 12:09:51.423249 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.423142 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret\") pod \"global-pull-secret-syncer-tcgfq\" (UID: \"88f19386-8325-4469-9859-cd876612dcd6\") " pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:09:51.423382 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:51.423261 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 12:09:51.423382 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:51.423311 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret podName:88f19386-8325-4469-9859-cd876612dcd6 nodeName:}" failed. No retries permitted until 2026-07-23 12:09:52.423298753 +0000 UTC m=+4.015833424 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret") pod "global-pull-secret-syncer-tcgfq" (UID: "88f19386-8325-4469-9859-cd876612dcd6") : object "kube-system"/"original-pull-secret" not registered Jul 23 12:09:51.524486 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.524131 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs\") pod \"network-metrics-daemon-cbm6t\" (UID: \"232bfbf8-288e-4679-8bee-4c5179866c00\") " pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:09:51.524486 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:51.524335 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 12:09:51.524486 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:51.524407 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs podName:232bfbf8-288e-4679-8bee-4c5179866c00 nodeName:}" failed. No retries permitted until 2026-07-23 12:09:52.524387348 +0000 UTC m=+4.116922036 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs") pod "network-metrics-daemon-cbm6t" (UID: "232bfbf8-288e-4679-8bee-4c5179866c00") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 12:09:51.625509 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.625475 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2rbr\" (UniqueName: \"kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr\") pod \"network-check-target-tjdx4\" (UID: \"6b308275-101d-4a54-86d4-fc63739354cb\") " pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:09:51.625679 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:51.625647 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 12:09:51.625679 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:51.625672 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 12:09:51.625793 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:51.625687 2578 projected.go:196] Error preparing data for projected volume kube-api-access-m2rbr for pod openshift-network-diagnostics/network-check-target-tjdx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 12:09:51.625793 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:51.625744 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr podName:6b308275-101d-4a54-86d4-fc63739354cb nodeName:}" failed. No retries permitted until 2026-07-23 12:09:52.6257262 +0000 UTC m=+4.218260875 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-m2rbr" (UniqueName: "kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr") pod "network-check-target-tjdx4" (UID: "6b308275-101d-4a54-86d4-fc63739354cb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 12:09:51.860393 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.860301 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 12:04:50 +0000 UTC" deadline="2028-02-29 20:29:07.314108595 +0000 UTC" Jul 23 12:09:51.860393 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.860347 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14072h19m15.453766278s" Jul 23 12:09:51.929020 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.928990 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:09:51.929220 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:51.929113 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcgfq" podUID="88f19386-8325-4469-9859-cd876612dcd6" Jul 23 12:09:51.929220 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.929187 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:09:51.929673 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:51.929638 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjdx4" podUID="6b308275-101d-4a54-86d4-fc63739354cb" Jul 23 12:09:51.959497 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.959457 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lzcnk" event={"ID":"cbede117-0113-4f86-a8b7-532164bf587f","Type":"ContainerStarted","Data":"a617f4441ebfa22809766f46bb987a34ceb2f774a2224b896fd6d2bf215d1d21"} Jul 23 12:09:51.962536 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.962426 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jl6fz" event={"ID":"3ee343ae-feea-4448-8ed4-641274b5be7e","Type":"ContainerStarted","Data":"7cfd1efb4cc6645000ff3cb02da3ab2a999de1d29bc9189b909b08b74eaba26a"} Jul 23 12:09:51.972842 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.972801 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fjtdt" event={"ID":"055376f5-9608-4f55-b8ae-e6f8b9d2329e","Type":"ContainerStarted","Data":"16e7229d3f4b8f34083f1cabe0377bed7c24bb4f2fd92c8e6ac98e6a41d4e686"} Jul 23 12:09:51.994721 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:51.994680 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" event={"ID":"024307b2-8735-4d0a-8673-0152c47b31bc","Type":"ContainerStarted","Data":"a17ef3552b67bf398060131b2bcad9a78b5a32f26863b132766e4737e755fb33"} Jul 23 12:09:52.016475 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:52.016403 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" event={"ID":"26f8cdf9-5ba7-4629-95a1-49f140dcecde","Type":"ContainerStarted","Data":"928407e07db5e17ffac56589b6cd1c597ed317c0439c96d6ca2e7e38e7127aa2"} Jul 23 12:09:52.040002 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:52.039965 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sdx4n" event={"ID":"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7","Type":"ContainerStarted","Data":"1a36ce2ef2596b7fce1b56dd34e5d8d1f893cbbfde31d48042011b5a7c35373f"} Jul 23 12:09:52.058501 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:52.058466 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rj4l2" event={"ID":"d3c0c881-554f-4572-bdaa-c0ff060c50ec","Type":"ContainerStarted","Data":"df7974e721bac0fa7d88d3db29522d1e0d8a156099b200cc3ba81fca8add7b11"} Jul 23 12:09:52.084323 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:52.083811 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zdc7h" event={"ID":"374f6cfe-ca3e-424f-9875-3040a3e78447","Type":"ContainerStarted","Data":"fbd77a930ee093d7bd0599c4167b0fef7da03505ba62930ad7a36708c15f4a9a"} Jul 23 12:09:52.098385 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:52.098350 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" event={"ID":"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6","Type":"ContainerStarted","Data":"f8ed430fff2eb9a190fb019ad91782ad8421f4a0e601e80224e7123027e5bfe3"} Jul 23 12:09:52.430998 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:52.430961 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret\") pod \"global-pull-secret-syncer-tcgfq\" (UID: \"88f19386-8325-4469-9859-cd876612dcd6\") " pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:09:52.443838 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:52.439409 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 12:09:52.443838 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:52.439524 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret podName:88f19386-8325-4469-9859-cd876612dcd6 nodeName:}" failed. No retries permitted until 2026-07-23 12:09:54.4394868 +0000 UTC m=+6.032021472 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret") pod "global-pull-secret-syncer-tcgfq" (UID: "88f19386-8325-4469-9859-cd876612dcd6") : object "kube-system"/"original-pull-secret" not registered Jul 23 12:09:52.532539 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:52.532495 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs\") pod \"network-metrics-daemon-cbm6t\" (UID: \"232bfbf8-288e-4679-8bee-4c5179866c00\") " pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:09:52.532735 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:52.532718 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 12:09:52.532813 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:52.532785 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs podName:232bfbf8-288e-4679-8bee-4c5179866c00 nodeName:}" failed. No retries permitted until 2026-07-23 12:09:54.532768003 +0000 UTC m=+6.125302689 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs") pod "network-metrics-daemon-cbm6t" (UID: "232bfbf8-288e-4679-8bee-4c5179866c00") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 12:09:52.634267 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:52.633587 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2rbr\" (UniqueName: \"kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr\") pod \"network-check-target-tjdx4\" (UID: \"6b308275-101d-4a54-86d4-fc63739354cb\") " pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:09:52.634267 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:52.633809 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 12:09:52.634267 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:52.633828 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 12:09:52.634267 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:52.633841 2578 projected.go:196] Error preparing data for projected volume kube-api-access-m2rbr for pod openshift-network-diagnostics/network-check-target-tjdx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 12:09:52.634267 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:52.633900 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr podName:6b308275-101d-4a54-86d4-fc63739354cb nodeName:}" failed. No retries permitted until 2026-07-23 12:09:54.633882605 +0000 UTC m=+6.226417282 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-m2rbr" (UniqueName: "kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr") pod "network-check-target-tjdx4" (UID: "6b308275-101d-4a54-86d4-fc63739354cb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 12:09:52.862414 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:52.862303 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 12:04:50 +0000 UTC" deadline="2028-01-15 05:44:39.197875244 +0000 UTC" Jul 23 12:09:52.862414 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:52.862370 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12977h34m46.335508884s" Jul 23 12:09:52.930733 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:52.930705 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:09:52.930909 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:52.930846 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbm6t" podUID="232bfbf8-288e-4679-8bee-4c5179866c00" Jul 23 12:09:53.929627 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:53.928929 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:09:53.929627 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:53.929060 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjdx4" podUID="6b308275-101d-4a54-86d4-fc63739354cb" Jul 23 12:09:53.929627 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:53.929465 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:09:53.929627 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:53.929562 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcgfq" podUID="88f19386-8325-4469-9859-cd876612dcd6" Jul 23 12:09:54.449794 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:54.449178 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret\") pod \"global-pull-secret-syncer-tcgfq\" (UID: \"88f19386-8325-4469-9859-cd876612dcd6\") " pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:09:54.449794 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:54.449382 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 12:09:54.449794 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:54.449447 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret podName:88f19386-8325-4469-9859-cd876612dcd6 nodeName:}" failed. No retries permitted until 2026-07-23 12:09:58.449427843 +0000 UTC m=+10.041962515 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret") pod "global-pull-secret-syncer-tcgfq" (UID: "88f19386-8325-4469-9859-cd876612dcd6") : object "kube-system"/"original-pull-secret" not registered Jul 23 12:09:54.550575 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:54.550540 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs\") pod \"network-metrics-daemon-cbm6t\" (UID: \"232bfbf8-288e-4679-8bee-4c5179866c00\") " pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:09:54.550784 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:54.550744 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 12:09:54.550912 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:54.550818 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs podName:232bfbf8-288e-4679-8bee-4c5179866c00 nodeName:}" failed. No retries permitted until 2026-07-23 12:09:58.550799358 +0000 UTC m=+10.143334030 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs") pod "network-metrics-daemon-cbm6t" (UID: "232bfbf8-288e-4679-8bee-4c5179866c00") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 12:09:54.651340 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:54.651302 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2rbr\" (UniqueName: \"kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr\") pod \"network-check-target-tjdx4\" (UID: \"6b308275-101d-4a54-86d4-fc63739354cb\") " pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:09:54.651508 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:54.651439 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 12:09:54.651508 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:54.651453 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 12:09:54.651508 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:54.651462 2578 projected.go:196] Error preparing data for projected volume kube-api-access-m2rbr for pod openshift-network-diagnostics/network-check-target-tjdx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 12:09:54.651687 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:54.651513 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr podName:6b308275-101d-4a54-86d4-fc63739354cb nodeName:}" failed. No retries permitted until 2026-07-23 12:09:58.651501114 +0000 UTC m=+10.244035784 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-m2rbr" (UniqueName: "kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr") pod "network-check-target-tjdx4" (UID: "6b308275-101d-4a54-86d4-fc63739354cb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 12:09:54.931354 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:54.931271 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:09:54.931792 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:54.931441 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbm6t" podUID="232bfbf8-288e-4679-8bee-4c5179866c00" Jul 23 12:09:55.928660 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:55.928630 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:09:55.928862 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:55.928745 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjdx4" podUID="6b308275-101d-4a54-86d4-fc63739354cb" Jul 23 12:09:55.929071 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:55.929050 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:09:55.929174 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:55.929153 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcgfq" podUID="88f19386-8325-4469-9859-cd876612dcd6" Jul 23 12:09:56.929082 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:56.929050 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:09:56.929500 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:56.929214 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbm6t" podUID="232bfbf8-288e-4679-8bee-4c5179866c00" Jul 23 12:09:57.928488 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:57.928411 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:09:57.928647 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:57.928547 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjdx4" podUID="6b308275-101d-4a54-86d4-fc63739354cb" Jul 23 12:09:57.928647 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:57.928588 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:09:57.928766 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:57.928706 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcgfq" podUID="88f19386-8325-4469-9859-cd876612dcd6" Jul 23 12:09:58.484540 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:58.484493 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret\") pod \"global-pull-secret-syncer-tcgfq\" (UID: \"88f19386-8325-4469-9859-cd876612dcd6\") " pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:09:58.485002 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:58.484643 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 12:09:58.485002 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:58.484734 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret podName:88f19386-8325-4469-9859-cd876612dcd6 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:06.484711179 +0000 UTC m=+18.077245872 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret") pod "global-pull-secret-syncer-tcgfq" (UID: "88f19386-8325-4469-9859-cd876612dcd6") : object "kube-system"/"original-pull-secret" not registered Jul 23 12:09:58.585500 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:58.585458 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs\") pod \"network-metrics-daemon-cbm6t\" (UID: \"232bfbf8-288e-4679-8bee-4c5179866c00\") " pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:09:58.585682 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:58.585599 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 12:09:58.585748 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:58.585686 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs podName:232bfbf8-288e-4679-8bee-4c5179866c00 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:06.585664159 +0000 UTC m=+18.178198833 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs") pod "network-metrics-daemon-cbm6t" (UID: "232bfbf8-288e-4679-8bee-4c5179866c00") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 12:09:58.686473 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:58.686433 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2rbr\" (UniqueName: \"kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr\") pod \"network-check-target-tjdx4\" (UID: \"6b308275-101d-4a54-86d4-fc63739354cb\") " pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:09:58.687064 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:58.686639 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 12:09:58.687064 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:58.686667 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 12:09:58.687064 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:58.686681 2578 projected.go:196] Error preparing data for projected volume kube-api-access-m2rbr for pod openshift-network-diagnostics/network-check-target-tjdx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 12:09:58.687064 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:58.686747 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr podName:6b308275-101d-4a54-86d4-fc63739354cb nodeName:}" failed. No retries permitted until 2026-07-23 12:10:06.686728097 +0000 UTC m=+18.279262771 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-m2rbr" (UniqueName: "kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr") pod "network-check-target-tjdx4" (UID: "6b308275-101d-4a54-86d4-fc63739354cb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 12:09:58.929664 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:58.929575 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:09:58.929824 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:58.929764 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbm6t" podUID="232bfbf8-288e-4679-8bee-4c5179866c00" Jul 23 12:09:59.928964 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:59.928920 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:09:59.928964 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:09:59.928947 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:09:59.929549 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:59.929057 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcgfq" podUID="88f19386-8325-4469-9859-cd876612dcd6" Jul 23 12:09:59.929549 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:09:59.929151 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjdx4" podUID="6b308275-101d-4a54-86d4-fc63739354cb" Jul 23 12:10:00.929139 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:00.928597 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:10:00.929139 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:00.928769 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbm6t" podUID="232bfbf8-288e-4679-8bee-4c5179866c00" Jul 23 12:10:01.928702 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:01.928657 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:10:01.928880 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:01.928657 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:10:01.928880 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:01.928782 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjdx4" podUID="6b308275-101d-4a54-86d4-fc63739354cb" Jul 23 12:10:01.928962 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:01.928885 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcgfq" podUID="88f19386-8325-4469-9859-cd876612dcd6" Jul 23 12:10:02.929048 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:02.928963 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:10:02.929541 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:02.929111 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbm6t" podUID="232bfbf8-288e-4679-8bee-4c5179866c00" Jul 23 12:10:03.928963 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:03.928921 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:10:03.929155 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:03.928924 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:10:03.929155 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:03.929034 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjdx4" podUID="6b308275-101d-4a54-86d4-fc63739354cb" Jul 23 12:10:03.929540 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:03.929147 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcgfq" podUID="88f19386-8325-4469-9859-cd876612dcd6" Jul 23 12:10:04.928491 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:04.928459 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:10:04.928669 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:04.928596 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbm6t" podUID="232bfbf8-288e-4679-8bee-4c5179866c00" Jul 23 12:10:05.928376 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:05.928337 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:10:05.928847 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:05.928337 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:10:05.928847 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:05.928540 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcgfq" podUID="88f19386-8325-4469-9859-cd876612dcd6" Jul 23 12:10:05.928847 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:05.928576 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjdx4" podUID="6b308275-101d-4a54-86d4-fc63739354cb" Jul 23 12:10:06.544175 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:06.544098 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret\") pod \"global-pull-secret-syncer-tcgfq\" (UID: \"88f19386-8325-4469-9859-cd876612dcd6\") " pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:10:06.544312 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:06.544280 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 12:10:06.544358 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:06.544348 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret podName:88f19386-8325-4469-9859-cd876612dcd6 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:22.544329957 +0000 UTC m=+34.136864645 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret") pod "global-pull-secret-syncer-tcgfq" (UID: "88f19386-8325-4469-9859-cd876612dcd6") : object "kube-system"/"original-pull-secret" not registered Jul 23 12:10:06.645409 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:06.645383 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs\") pod \"network-metrics-daemon-cbm6t\" (UID: \"232bfbf8-288e-4679-8bee-4c5179866c00\") " pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:10:06.645541 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:06.645523 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 12:10:06.645591 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:06.645582 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs podName:232bfbf8-288e-4679-8bee-4c5179866c00 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:22.645566895 +0000 UTC m=+34.238101571 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs") pod "network-metrics-daemon-cbm6t" (UID: "232bfbf8-288e-4679-8bee-4c5179866c00") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 12:10:06.745834 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:06.745785 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2rbr\" (UniqueName: \"kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr\") pod \"network-check-target-tjdx4\" (UID: \"6b308275-101d-4a54-86d4-fc63739354cb\") " pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:10:06.745996 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:06.745967 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 12:10:06.745996 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:06.745994 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 12:10:06.746114 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:06.746004 2578 projected.go:196] Error preparing data for projected volume kube-api-access-m2rbr for pod openshift-network-diagnostics/network-check-target-tjdx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 12:10:06.746114 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:06.746062 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr podName:6b308275-101d-4a54-86d4-fc63739354cb nodeName:}" failed. No retries permitted until 2026-07-23 12:10:22.746043719 +0000 UTC m=+34.338578395 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-m2rbr" (UniqueName: "kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr") pod "network-check-target-tjdx4" (UID: "6b308275-101d-4a54-86d4-fc63739354cb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 12:10:06.928580 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:06.928487 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:10:06.929038 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:06.928650 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbm6t" podUID="232bfbf8-288e-4679-8bee-4c5179866c00" Jul 23 12:10:07.929222 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:07.929171 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:10:07.929678 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:07.929312 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjdx4" podUID="6b308275-101d-4a54-86d4-fc63739354cb" Jul 23 12:10:07.929678 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:07.929360 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:10:07.929678 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:07.929479 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcgfq" podUID="88f19386-8325-4469-9859-cd876612dcd6" Jul 23 12:10:08.929444 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:08.929402 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:10:08.929890 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:08.929522 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbm6t" podUID="232bfbf8-288e-4679-8bee-4c5179866c00" Jul 23 12:10:09.928858 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:09.928829 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:10:09.929053 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:09.928830 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:10:09.929053 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:09.929011 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcgfq" podUID="88f19386-8325-4469-9859-cd876612dcd6" Jul 23 12:10:09.929137 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:09.929088 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjdx4" podUID="6b308275-101d-4a54-86d4-fc63739354cb" Jul 23 12:10:10.928969 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:10.928944 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:10:10.929305 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:10.929054 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbm6t" podUID="232bfbf8-288e-4679-8bee-4c5179866c00" Jul 23 12:10:11.153432 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:11.153213 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal" event={"ID":"542075cdbfd82dba7e896ceb37827ec4","Type":"ContainerStarted","Data":"860c3a1f01a29ff277c1c2b4093f5928a4d51bb5edb817cab2e10277a1d4e0ba"} Jul 23 12:10:11.929064 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:11.929039 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:10:11.929438 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:11.929079 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:10:11.929438 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:11.929166 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcgfq" podUID="88f19386-8325-4469-9859-cd876612dcd6" Jul 23 12:10:11.929438 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:11.929267 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjdx4" podUID="6b308275-101d-4a54-86d4-fc63739354cb" Jul 23 12:10:12.160158 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.160124 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sdx4n" event={"ID":"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7","Type":"ContainerStarted","Data":"3402a07d1a5a9d6170a5239af2bbc18112a2b0c8af0d6fa41854b372e6370e27"} Jul 23 12:10:12.161896 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.161864 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rj4l2" event={"ID":"d3c0c881-554f-4572-bdaa-c0ff060c50ec","Type":"ContainerStarted","Data":"1da971f6df85d79536fc34ee5340f2ef5807d65cbe377b809ac32939821154de"} Jul 23 12:10:12.163376 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.163350 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zdc7h" event={"ID":"374f6cfe-ca3e-424f-9875-3040a3e78447","Type":"ContainerStarted","Data":"e0dc0fbb905ee571efa24984b2824b059b0cc165a9eb42a6f7b976bd4a6196c3"} Jul 23 12:10:12.164766 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.164747 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" event={"ID":"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6","Type":"ContainerStarted","Data":"97d4d8b36259657cc1e5a11b8948b4d86e0e109583e6b7c243f5a98eeea056b8"} Jul 23 12:10:12.165938 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.165918 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jl6fz" event={"ID":"3ee343ae-feea-4448-8ed4-641274b5be7e","Type":"ContainerStarted","Data":"ea7d55a23ae985069cdca66fa137b3120e3de93a8b95403de3ba14557add1ea5"} Jul 23 12:10:12.167216 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.167182 2578 generic.go:356] "Generic (PLEG): container finished" podID="5fb9e113ed4aa68386e433953fb9a7e6" containerID="2309d9c216fa1e08dd984398d751b1a4f0253d1d35fe01823115b106e95242f6" exitCode=0 Jul 23 12:10:12.167296 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.167218 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal" event={"ID":"5fb9e113ed4aa68386e433953fb9a7e6","Type":"ContainerDied","Data":"2309d9c216fa1e08dd984398d751b1a4f0253d1d35fe01823115b106e95242f6"} Jul 23 12:10:12.168630 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.168612 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fjtdt" event={"ID":"055376f5-9608-4f55-b8ae-e6f8b9d2329e","Type":"ContainerStarted","Data":"e814f219ba8678ed5f066631bceffc6aca6bd6f3b513c5d2feb122fb446010c2"} Jul 23 12:10:12.170341 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.170323 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:10:12.170625 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.170608 2578 generic.go:356] "Generic (PLEG): container finished" podID="024307b2-8735-4d0a-8673-0152c47b31bc" containerID="476ddf5f662a851b30d3c7c15f820cdd50d4fa750367b1670597ab51ea39f3fb" exitCode=1 Jul 23 12:10:12.170680 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.170659 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" event={"ID":"024307b2-8735-4d0a-8673-0152c47b31bc","Type":"ContainerStarted","Data":"2d2a77b03a90f32a90e9d6a79b0d184fd9f9aee393dfae6c381bf05b863c00bb"} Jul 23 12:10:12.170727 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.170714 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" event={"ID":"024307b2-8735-4d0a-8673-0152c47b31bc","Type":"ContainerDied","Data":"476ddf5f662a851b30d3c7c15f820cdd50d4fa750367b1670597ab51ea39f3fb"} Jul 23 12:10:12.170773 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.170756 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" event={"ID":"024307b2-8735-4d0a-8673-0152c47b31bc","Type":"ContainerStarted","Data":"4415a582551256c99177a032de14724c09cc7b83916d4806d16183b1b2a1842c"} Jul 23 12:10:12.171833 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.171805 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" event={"ID":"26f8cdf9-5ba7-4629-95a1-49f140dcecde","Type":"ContainerStarted","Data":"1e07ac97f884a3a5f799d78da9e12cce107aec229d49bddf628ebe0259e689ca"} Jul 23 12:10:12.218882 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.218827 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-37.ec2.internal" podStartSLOduration=22.218807666 podStartE2EDuration="22.218807666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:09:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:12.217891561 +0000 UTC m=+23.810426253" watchObservedRunningTime="2026-07-23 12:10:12.218807666 +0000 UTC m=+23.811342360" Jul 23 12:10:12.235365 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.235175 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hwrnq" podStartSLOduration=3.448901388 podStartE2EDuration="23.235154311s" totalImagesPullingTime="19.786252923s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:09:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:12.234868152 +0000 UTC m=+23.827402846" watchObservedRunningTime="2026-07-23 12:10:12.235154311 +0000 UTC m=+23.827689007" Jul 23 12:10:12.250397 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.250363 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jl6fz" podStartSLOduration=8.11000653 podStartE2EDuration="23.250351322s" totalImagesPullingTime="15.140344792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:09:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:12.250284845 +0000 UTC m=+23.842819537" watchObservedRunningTime="2026-07-23 12:10:12.250351322 +0000 UTC m=+23.842886014" Jul 23 12:10:12.293668 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.293619 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rj4l2" podStartSLOduration=3.537803307 podStartE2EDuration="23.293599313s" totalImagesPullingTime="19.755796006s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:09:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:12.293187455 +0000 UTC m=+23.885722150" watchObservedRunningTime="2026-07-23 12:10:12.293599313 +0000 UTC m=+23.886134007" Jul 23 12:10:12.293759 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.293719 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zdc7h" podStartSLOduration=7.81654551 podStartE2EDuration="23.293711206s" totalImagesPullingTime="15.477165696s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:09:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:12.271710958 +0000 UTC m=+23.864245651" watchObservedRunningTime="2026-07-23 12:10:12.293711206 +0000 UTC m=+23.886245900" Jul 23 12:10:12.313007 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.312641 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fjtdt" podStartSLOduration=3.265644657 podStartE2EDuration="23.312623876s" totalImagesPullingTime="20.046979219s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:09:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:12.312067747 +0000 UTC m=+23.904602441" watchObservedRunningTime="2026-07-23 12:10:12.312623876 +0000 UTC m=+23.905158570" Jul 23 12:10:12.928703 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:12.928660 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:10:12.928872 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:12.928820 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbm6t" podUID="232bfbf8-288e-4679-8bee-4c5179866c00" Jul 23 12:10:13.175510 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:13.175476 2578 generic.go:356] "Generic (PLEG): container finished" podID="52f44df7-1c9e-4e78-9c0a-b989bf1f70c7" containerID="3402a07d1a5a9d6170a5239af2bbc18112a2b0c8af0d6fa41854b372e6370e27" exitCode=0 Jul 23 12:10:13.176343 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:13.175527 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sdx4n" event={"ID":"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7","Type":"ContainerDied","Data":"3402a07d1a5a9d6170a5239af2bbc18112a2b0c8af0d6fa41854b372e6370e27"} Jul 23 12:10:13.176946 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:13.176920 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lzcnk" event={"ID":"cbede117-0113-4f86-a8b7-532164bf587f","Type":"ContainerStarted","Data":"56a099997c6d532f69d37cf630472cfde31f01d6fdb7bab91109865469997618"} Jul 23 12:10:13.178479 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:13.178457 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal" event={"ID":"5fb9e113ed4aa68386e433953fb9a7e6","Type":"ContainerStarted","Data":"496f03ed14fbb136b24bf84cb83a77e40ac7dcd1f6488bed803a6a06a7dfd3c8"} Jul 23 12:10:13.181143 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:13.181098 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:10:13.181565 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:13.181475 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" event={"ID":"024307b2-8735-4d0a-8673-0152c47b31bc","Type":"ContainerStarted","Data":"23e44eed48be6242838fc4bc07bd5a6bcc6452c2f11392e7201130c8ca4f4d32"} Jul 23 12:10:13.181565 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:13.181550 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" event={"ID":"024307b2-8735-4d0a-8673-0152c47b31bc","Type":"ContainerStarted","Data":"6ce0724b858b71fa590b0e04a853d0e104d0db0087eeecca637ea9af5eaa1c28"} Jul 23 12:10:13.181702 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:13.181593 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" event={"ID":"024307b2-8735-4d0a-8673-0152c47b31bc","Type":"ContainerStarted","Data":"3ef680c008cfffda08942b1b1be2cee0a5985b8cb55b58a928045327cfba1f2e"} Jul 23 12:10:13.217993 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:13.217945 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-37.ec2.internal" podStartSLOduration=23.217930219 podStartE2EDuration="23.217930219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:09:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:13.217878138 +0000 UTC m=+24.810412833" watchObservedRunningTime="2026-07-23 12:10:13.217930219 +0000 UTC m=+24.810464913" Jul 23 12:10:13.234691 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:13.234652 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lzcnk" podStartSLOduration=4.468115443 podStartE2EDuration="24.234637438s" totalImagesPullingTime="19.766521995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:09:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:13.234166063 +0000 UTC m=+24.826700755" watchObservedRunningTime="2026-07-23 12:10:13.234637438 +0000 UTC m=+24.827172131" Jul 23 12:10:13.336050 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:13.336027 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 23 12:10:13.895459 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:13.895345 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T12:10:13.336042791Z","UUID":"33a3ac26-0816-4667-b4ce-7fa729fc4152","Handler":null,"Name":"","Endpoint":""} Jul 23 12:10:13.896946 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:13.896921 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 23 12:10:13.897076 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:13.896963 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 23 12:10:13.928433 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:13.928398 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:10:13.928588 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:13.928443 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:10:13.928588 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:13.928526 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcgfq" podUID="88f19386-8325-4469-9859-cd876612dcd6" Jul 23 12:10:13.928720 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:13.928633 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjdx4" podUID="6b308275-101d-4a54-86d4-fc63739354cb" Jul 23 12:10:14.185547 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:14.185459 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" event={"ID":"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6","Type":"ContainerStarted","Data":"70df2815d4e5e016aa2aee353d0de960527fee5c4e8a805c632c0bba3a483324"} Jul 23 12:10:14.221367 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:14.221226 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jl6fz" Jul 23 12:10:14.221894 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:14.221871 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jl6fz" Jul 23 12:10:14.929279 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:14.928989 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:10:14.929425 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:14.929360 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbm6t" podUID="232bfbf8-288e-4679-8bee-4c5179866c00" Jul 23 12:10:15.191482 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:15.191404 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:10:15.191853 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:15.191808 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" event={"ID":"024307b2-8735-4d0a-8673-0152c47b31bc","Type":"ContainerStarted","Data":"60edb9229fc8d2c8afd332ce651b73263799aa1d80280fafafbd46ffbf9a22dc"} Jul 23 12:10:15.194203 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:15.194160 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" event={"ID":"f4bf1809-2e30-4f4e-8e1d-059a5d44bec6","Type":"ContainerStarted","Data":"586d5998b8ee5622f9f7d9249111fc836d510ebe8da6436f9067e204dbe84929"} Jul 23 12:10:15.194640 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:15.194558 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jl6fz" Jul 23 12:10:15.195074 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:15.195057 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jl6fz" Jul 23 12:10:15.216093 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:15.216044 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p55js" podStartSLOduration=4.200768056 podStartE2EDuration="26.2160242s" totalImagesPullingTime="22.015256144s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:09:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:15.215185766 +0000 UTC m=+26.807720459" watchObservedRunningTime="2026-07-23 12:10:15.2160242 +0000 UTC m=+26.808558894" Jul 23 12:10:15.929225 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:15.929177 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:10:15.929379 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:15.929180 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:10:15.929379 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:15.929288 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjdx4" podUID="6b308275-101d-4a54-86d4-fc63739354cb" Jul 23 12:10:15.929379 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:15.929358 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcgfq" podUID="88f19386-8325-4469-9859-cd876612dcd6" Jul 23 12:10:16.928963 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:16.928929 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:10:16.929581 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:16.929054 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbm6t" podUID="232bfbf8-288e-4679-8bee-4c5179866c00" Jul 23 12:10:17.929369 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:17.929227 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:10:17.929817 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:17.929266 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:10:17.929817 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:17.929550 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcgfq" podUID="88f19386-8325-4469-9859-cd876612dcd6" Jul 23 12:10:17.929817 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:17.929438 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjdx4" podUID="6b308275-101d-4a54-86d4-fc63739354cb" Jul 23 12:10:18.204607 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:18.204531 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:10:18.204912 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:18.204891 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" event={"ID":"024307b2-8735-4d0a-8673-0152c47b31bc","Type":"ContainerStarted","Data":"5c1852113930fd930c9747e1501606ba65c7c56c979d4e2fe6a1833f9230150c"} Jul 23 12:10:18.205159 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:18.205119 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:10:18.205296 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:18.205205 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:10:18.205360 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:18.205347 2578 scope.go:117] "RemoveContainer" containerID="476ddf5f662a851b30d3c7c15f820cdd50d4fa750367b1670597ab51ea39f3fb" Jul 23 12:10:18.206886 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:18.206859 2578 generic.go:356] "Generic (PLEG): container finished" podID="52f44df7-1c9e-4e78-9c0a-b989bf1f70c7" containerID="164f56b89b6bad317c9e9fe56c498237ef60746002dd07b0c7b02fbf506e8c5a" exitCode=0 Jul 23 12:10:18.206975 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:18.206899 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sdx4n" event={"ID":"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7","Type":"ContainerDied","Data":"164f56b89b6bad317c9e9fe56c498237ef60746002dd07b0c7b02fbf506e8c5a"} Jul 23 12:10:18.221236 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:18.221215 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:10:18.940756 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:18.940731 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:10:18.941316 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:18.940840 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbm6t" podUID="232bfbf8-288e-4679-8bee-4c5179866c00" Jul 23 12:10:19.213010 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:19.212944 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:10:19.213317 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:19.213292 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" event={"ID":"024307b2-8735-4d0a-8673-0152c47b31bc","Type":"ContainerStarted","Data":"e1ab61d8b5748564b39f9ddaa51d572ba490fc75cae260a1ebe6f580799afe20"} Jul 23 12:10:19.213485 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:19.213466 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:10:19.228863 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:19.228837 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:10:19.260516 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:19.260460 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" podStartSLOduration=9.600795 podStartE2EDuration="30.260445833s" totalImagesPullingTime="20.659650833s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:09:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:19.256966266 +0000 UTC m=+30.849500961" watchObservedRunningTime="2026-07-23 12:10:19.260445833 +0000 UTC m=+30.852980526" Jul 23 12:10:19.817618 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:19.817540 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tcgfq"] Jul 23 12:10:19.817750 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:19.817652 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:10:19.817750 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:19.817730 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcgfq" podUID="88f19386-8325-4469-9859-cd876612dcd6" Jul 23 12:10:19.820208 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:19.820175 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tjdx4"] Jul 23 12:10:19.820305 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:19.820277 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:10:19.820374 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:19.820355 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjdx4" podUID="6b308275-101d-4a54-86d4-fc63739354cb" Jul 23 12:10:19.821653 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:19.821629 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cbm6t"] Jul 23 12:10:19.821738 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:19.821716 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:10:19.821817 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:19.821797 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbm6t" podUID="232bfbf8-288e-4679-8bee-4c5179866c00" Jul 23 12:10:20.217637 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:20.217548 2578 generic.go:356] "Generic (PLEG): container finished" podID="52f44df7-1c9e-4e78-9c0a-b989bf1f70c7" containerID="531005df29d9cae3bb4da7b5e925981d45847f7dae742657809dba091f476afc" exitCode=0 Jul 23 12:10:20.218109 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:20.217631 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sdx4n" event={"ID":"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7","Type":"ContainerDied","Data":"531005df29d9cae3bb4da7b5e925981d45847f7dae742657809dba091f476afc"} Jul 23 12:10:21.929508 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:21.929322 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:10:21.930132 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:21.929370 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:10:21.930132 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:21.929595 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjdx4" podUID="6b308275-101d-4a54-86d4-fc63739354cb" Jul 23 12:10:21.930132 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:21.929371 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:10:21.930132 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:21.929653 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbm6t" podUID="232bfbf8-288e-4679-8bee-4c5179866c00" Jul 23 12:10:21.930132 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:21.929710 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcgfq" podUID="88f19386-8325-4469-9859-cd876612dcd6" Jul 23 12:10:22.223626 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:22.223593 2578 generic.go:356] "Generic (PLEG): container finished" podID="52f44df7-1c9e-4e78-9c0a-b989bf1f70c7" containerID="8a1249e67a8158da022d865205d2c7e27b0ea974ef39eeed1930b6f010a91498" exitCode=0 Jul 23 12:10:22.223795 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:22.223658 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sdx4n" event={"ID":"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7","Type":"ContainerDied","Data":"8a1249e67a8158da022d865205d2c7e27b0ea974ef39eeed1930b6f010a91498"} Jul 23 12:10:22.567883 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:22.567805 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret\") pod \"global-pull-secret-syncer-tcgfq\" (UID: \"88f19386-8325-4469-9859-cd876612dcd6\") " pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:10:22.568045 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:22.567919 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 12:10:22.568045 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:22.567969 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret podName:88f19386-8325-4469-9859-cd876612dcd6 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:54.56795598 +0000 UTC m=+66.160490651 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret") pod "global-pull-secret-syncer-tcgfq" (UID: "88f19386-8325-4469-9859-cd876612dcd6") : object "kube-system"/"original-pull-secret" not registered Jul 23 12:10:22.668615 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:22.668583 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs\") pod \"network-metrics-daemon-cbm6t\" (UID: \"232bfbf8-288e-4679-8bee-4c5179866c00\") " pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:10:22.668794 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:22.668741 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 12:10:22.668841 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:22.668809 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs podName:232bfbf8-288e-4679-8bee-4c5179866c00 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:54.668791419 +0000 UTC m=+66.261326090 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs") pod "network-metrics-daemon-cbm6t" (UID: "232bfbf8-288e-4679-8bee-4c5179866c00") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 12:10:22.769528 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:22.769491 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2rbr\" (UniqueName: \"kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr\") pod \"network-check-target-tjdx4\" (UID: \"6b308275-101d-4a54-86d4-fc63739354cb\") " pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:10:22.769693 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:22.769671 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 12:10:22.769739 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:22.769696 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 12:10:22.769739 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:22.769709 2578 projected.go:196] Error preparing data for projected volume kube-api-access-m2rbr for pod openshift-network-diagnostics/network-check-target-tjdx4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 12:10:22.769837 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:22.769769 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr podName:6b308275-101d-4a54-86d4-fc63739354cb nodeName:}" failed. No retries permitted until 2026-07-23 12:10:54.769750095 +0000 UTC m=+66.362284782 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-m2rbr" (UniqueName: "kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr") pod "network-check-target-tjdx4" (UID: "6b308275-101d-4a54-86d4-fc63739354cb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 12:10:23.928871 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:23.928836 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:10:23.929462 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:23.928836 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:10:23.929462 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:23.928970 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:10:23.929462 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:23.928945 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tjdx4" podUID="6b308275-101d-4a54-86d4-fc63739354cb" Jul 23 12:10:23.929462 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:23.929066 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cbm6t" podUID="232bfbf8-288e-4679-8bee-4c5179866c00" Jul 23 12:10:23.929462 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:23.929143 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcgfq" podUID="88f19386-8325-4469-9859-cd876612dcd6" Jul 23 12:10:25.248174 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.248098 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-37.ec2.internal" event="NodeReady" Jul 23 12:10:25.248685 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.248279 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 23 12:10:25.287520 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.287486 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5999b54cbc-wg5fw"] Jul 23 12:10:25.290221 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.290187 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.293128 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.293105 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 23 12:10:25.293439 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.293415 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 23 12:10:25.293522 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.293457 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xdbs4\"" Jul 23 12:10:25.293790 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.293760 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 23 12:10:25.299762 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.299745 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 23 12:10:25.304725 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.304702 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5999b54cbc-wg5fw"] Jul 23 12:10:25.307437 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.307415 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2wl6p"] Jul 23 12:10:25.309172 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.309152 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ffzsb"] Jul 23 12:10:25.309303 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.309285 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2wl6p" Jul 23 12:10:25.313096 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.311409 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ffzsb" Jul 23 12:10:25.313096 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.312094 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 23 12:10:25.313096 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.312360 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 23 12:10:25.313096 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.312585 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dbmrb\"" Jul 23 12:10:25.313760 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.313740 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 23 12:10:25.314168 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.314143 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 23 12:10:25.314350 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.314183 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 23 12:10:25.314350 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.314268 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2gntm\"" Jul 23 12:10:25.328616 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.328588 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ffzsb"] Jul 23 12:10:25.329179 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.329152 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2wl6p"] Jul 23 12:10:25.389551 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.389513 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-certificates\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.389551 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.389557 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt2b9\" (UniqueName: \"kubernetes.io/projected/f8597295-ae1a-42e6-ac3f-293915f8a69c-kube-api-access-jt2b9\") pod \"dns-default-2wl6p\" (UID: \"f8597295-ae1a-42e6-ac3f-293915f8a69c\") " pod="openshift-dns/dns-default-2wl6p" Jul 23 12:10:25.389771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.389584 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-trusted-ca\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.389771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.389665 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-image-registry-private-configuration\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.389771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.389694 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.389771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.389722 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8597295-ae1a-42e6-ac3f-293915f8a69c-tmp-dir\") pod \"dns-default-2wl6p\" (UID: \"f8597295-ae1a-42e6-ac3f-293915f8a69c\") " pod="openshift-dns/dns-default-2wl6p" Jul 23 12:10:25.389938 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.389783 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b8vf\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-kube-api-access-5b8vf\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.389938 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.389837 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f8597295-ae1a-42e6-ac3f-293915f8a69c-config-volume\") pod \"dns-default-2wl6p\" (UID: \"f8597295-ae1a-42e6-ac3f-293915f8a69c\") " pod="openshift-dns/dns-default-2wl6p" Jul 23 12:10:25.389938 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.389863 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-ca-trust-extracted\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.389938 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.389889 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-bound-sa-token\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.389938 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.389918 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls\") pod \"dns-default-2wl6p\" (UID: \"f8597295-ae1a-42e6-ac3f-293915f8a69c\") " pod="openshift-dns/dns-default-2wl6p" Jul 23 12:10:25.390118 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.389946 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-installation-pull-secrets\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.490896 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.490856 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b8vf\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-kube-api-access-5b8vf\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.490896 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.490900 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f8597295-ae1a-42e6-ac3f-293915f8a69c-config-volume\") pod \"dns-default-2wl6p\" (UID: \"f8597295-ae1a-42e6-ac3f-293915f8a69c\") " pod="openshift-dns/dns-default-2wl6p" Jul 23 12:10:25.491127 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.490925 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-ca-trust-extracted\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.491127 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.490948 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-bound-sa-token\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.491127 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.490978 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d13b8306-c63d-4eaa-92be-8ba79051dbd3-cert\") pod \"ingress-canary-ffzsb\" (UID: \"d13b8306-c63d-4eaa-92be-8ba79051dbd3\") " pod="openshift-ingress-canary/ingress-canary-ffzsb" Jul 23 12:10:25.491127 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.491003 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls\") pod \"dns-default-2wl6p\" (UID: \"f8597295-ae1a-42e6-ac3f-293915f8a69c\") " pod="openshift-dns/dns-default-2wl6p" Jul 23 12:10:25.491127 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.491025 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-installation-pull-secrets\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.491127 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.491111 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr292\" (UniqueName: \"kubernetes.io/projected/d13b8306-c63d-4eaa-92be-8ba79051dbd3-kube-api-access-dr292\") pod \"ingress-canary-ffzsb\" (UID: \"d13b8306-c63d-4eaa-92be-8ba79051dbd3\") " pod="openshift-ingress-canary/ingress-canary-ffzsb" Jul 23 12:10:25.491456 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.491156 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-certificates\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.491456 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.491207 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt2b9\" (UniqueName: \"kubernetes.io/projected/f8597295-ae1a-42e6-ac3f-293915f8a69c-kube-api-access-jt2b9\") pod \"dns-default-2wl6p\" (UID: \"f8597295-ae1a-42e6-ac3f-293915f8a69c\") " pod="openshift-dns/dns-default-2wl6p" Jul 23 12:10:25.491456 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.491236 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-trusted-ca\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.491456 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.491317 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-image-registry-private-configuration\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.491456 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.491352 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.491456 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.491401 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-ca-trust-extracted\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.491740 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:25.491535 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 12:10:25.491740 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:25.491553 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5999b54cbc-wg5fw: secret "image-registry-tls" not found Jul 23 12:10:25.491740 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.491554 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8597295-ae1a-42e6-ac3f-293915f8a69c-tmp-dir\") pod \"dns-default-2wl6p\" (UID: \"f8597295-ae1a-42e6-ac3f-293915f8a69c\") " pod="openshift-dns/dns-default-2wl6p" Jul 23 12:10:25.491740 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:25.491615 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls podName:c3e44bef-488e-4d15-bd18-6cbbeec2b5eb nodeName:}" failed. No retries permitted until 2026-07-23 12:10:25.99159535 +0000 UTC m=+37.584130035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls") pod "image-registry-5999b54cbc-wg5fw" (UID: "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb") : secret "image-registry-tls" not found Jul 23 12:10:25.491740 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.491705 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f8597295-ae1a-42e6-ac3f-293915f8a69c-config-volume\") pod \"dns-default-2wl6p\" (UID: \"f8597295-ae1a-42e6-ac3f-293915f8a69c\") " pod="openshift-dns/dns-default-2wl6p" Jul 23 12:10:25.491987 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:25.491824 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 12:10:25.491987 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.491831 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f8597295-ae1a-42e6-ac3f-293915f8a69c-tmp-dir\") pod \"dns-default-2wl6p\" (UID: \"f8597295-ae1a-42e6-ac3f-293915f8a69c\") " pod="openshift-dns/dns-default-2wl6p" Jul 23 12:10:25.491987 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:25.491889 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls podName:f8597295-ae1a-42e6-ac3f-293915f8a69c nodeName:}" failed. No retries permitted until 2026-07-23 12:10:25.991872617 +0000 UTC m=+37.584407292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls") pod "dns-default-2wl6p" (UID: "f8597295-ae1a-42e6-ac3f-293915f8a69c") : secret "dns-default-metrics-tls" not found Jul 23 12:10:25.491987 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.491891 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-certificates\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.492258 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.492237 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-trusted-ca\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.495736 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.495708 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-installation-pull-secrets\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.496100 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.496078 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-image-registry-private-configuration\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.502637 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.502577 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt2b9\" (UniqueName: \"kubernetes.io/projected/f8597295-ae1a-42e6-ac3f-293915f8a69c-kube-api-access-jt2b9\") pod \"dns-default-2wl6p\" (UID: \"f8597295-ae1a-42e6-ac3f-293915f8a69c\") " pod="openshift-dns/dns-default-2wl6p" Jul 23 12:10:25.502637 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.502578 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-bound-sa-token\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.502805 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.502665 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b8vf\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-kube-api-access-5b8vf\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.592704 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.592643 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d13b8306-c63d-4eaa-92be-8ba79051dbd3-cert\") pod \"ingress-canary-ffzsb\" (UID: \"d13b8306-c63d-4eaa-92be-8ba79051dbd3\") " pod="openshift-ingress-canary/ingress-canary-ffzsb" Jul 23 12:10:25.592873 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.592730 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr292\" (UniqueName: \"kubernetes.io/projected/d13b8306-c63d-4eaa-92be-8ba79051dbd3-kube-api-access-dr292\") pod \"ingress-canary-ffzsb\" (UID: \"d13b8306-c63d-4eaa-92be-8ba79051dbd3\") " pod="openshift-ingress-canary/ingress-canary-ffzsb" Jul 23 12:10:25.595626 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.595598 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d13b8306-c63d-4eaa-92be-8ba79051dbd3-cert\") pod \"ingress-canary-ffzsb\" (UID: \"d13b8306-c63d-4eaa-92be-8ba79051dbd3\") " pod="openshift-ingress-canary/ingress-canary-ffzsb" Jul 23 12:10:25.610743 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.610717 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr292\" (UniqueName: \"kubernetes.io/projected/d13b8306-c63d-4eaa-92be-8ba79051dbd3-kube-api-access-dr292\") pod \"ingress-canary-ffzsb\" (UID: \"d13b8306-c63d-4eaa-92be-8ba79051dbd3\") " pod="openshift-ingress-canary/ingress-canary-ffzsb" Jul 23 12:10:25.629595 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.629566 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ffzsb" Jul 23 12:10:25.928839 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.928806 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:10:25.929019 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.928844 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:10:25.929019 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.928806 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:10:25.934574 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.934524 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 12:10:25.934704 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.934575 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k5z96\"" Jul 23 12:10:25.934704 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.934598 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 12:10:25.934704 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.934634 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pbg55\"" Jul 23 12:10:25.934704 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.934575 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 12:10:25.934704 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.934675 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 12:10:25.996103 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.996062 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:25.996295 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:25.996132 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls\") pod \"dns-default-2wl6p\" (UID: \"f8597295-ae1a-42e6-ac3f-293915f8a69c\") " pod="openshift-dns/dns-default-2wl6p" Jul 23 12:10:25.996295 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:25.996229 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 12:10:25.996295 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:25.996281 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls podName:f8597295-ae1a-42e6-ac3f-293915f8a69c nodeName:}" failed. No retries permitted until 2026-07-23 12:10:26.996267677 +0000 UTC m=+38.588802348 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls") pod "dns-default-2wl6p" (UID: "f8597295-ae1a-42e6-ac3f-293915f8a69c") : secret "dns-default-metrics-tls" not found Jul 23 12:10:25.996462 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:25.996234 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 12:10:25.996462 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:25.996347 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5999b54cbc-wg5fw: secret "image-registry-tls" not found Jul 23 12:10:25.996462 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:25.996411 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls podName:c3e44bef-488e-4d15-bd18-6cbbeec2b5eb nodeName:}" failed. No retries permitted until 2026-07-23 12:10:26.996393396 +0000 UTC m=+38.588928070 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls") pod "image-registry-5999b54cbc-wg5fw" (UID: "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb") : secret "image-registry-tls" not found Jul 23 12:10:26.382229 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:26.382186 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ffzsb"] Jul 23 12:10:26.385688 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:10:26.385662 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd13b8306_c63d_4eaa_92be_8ba79051dbd3.slice/crio-f42d18d3f8bd3d0fef788c7db86cfb2a85475575e9cea77978f4acbe9a53ad18 WatchSource:0}: Error finding container f42d18d3f8bd3d0fef788c7db86cfb2a85475575e9cea77978f4acbe9a53ad18: Status 404 returned error can't find the container with id f42d18d3f8bd3d0fef788c7db86cfb2a85475575e9cea77978f4acbe9a53ad18 Jul 23 12:10:27.004911 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:27.004877 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:27.005089 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:27.004926 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls\") pod \"dns-default-2wl6p\" (UID: \"f8597295-ae1a-42e6-ac3f-293915f8a69c\") " pod="openshift-dns/dns-default-2wl6p" Jul 23 12:10:27.005089 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:27.005016 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 12:10:27.005089 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:27.005034 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 12:10:27.005089 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:27.005053 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5999b54cbc-wg5fw: secret "image-registry-tls" not found Jul 23 12:10:27.005089 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:27.005066 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls podName:f8597295-ae1a-42e6-ac3f-293915f8a69c nodeName:}" failed. No retries permitted until 2026-07-23 12:10:29.005053551 +0000 UTC m=+40.597588222 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls") pod "dns-default-2wl6p" (UID: "f8597295-ae1a-42e6-ac3f-293915f8a69c") : secret "dns-default-metrics-tls" not found Jul 23 12:10:27.005304 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:27.005094 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls podName:c3e44bef-488e-4d15-bd18-6cbbeec2b5eb nodeName:}" failed. No retries permitted until 2026-07-23 12:10:29.00508268 +0000 UTC m=+40.597617352 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls") pod "image-registry-5999b54cbc-wg5fw" (UID: "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb") : secret "image-registry-tls" not found Jul 23 12:10:27.238822 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:27.238782 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ffzsb" event={"ID":"d13b8306-c63d-4eaa-92be-8ba79051dbd3","Type":"ContainerStarted","Data":"f42d18d3f8bd3d0fef788c7db86cfb2a85475575e9cea77978f4acbe9a53ad18"} Jul 23 12:10:27.242345 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:27.242310 2578 generic.go:356] "Generic (PLEG): container finished" podID="52f44df7-1c9e-4e78-9c0a-b989bf1f70c7" containerID="a6fb59b65585f15bf288e6fa1e310bd109fa941fee0b43f38971b0d3711a323b" exitCode=0 Jul 23 12:10:27.242345 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:27.242351 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sdx4n" event={"ID":"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7","Type":"ContainerDied","Data":"a6fb59b65585f15bf288e6fa1e310bd109fa941fee0b43f38971b0d3711a323b"} Jul 23 12:10:28.248141 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:28.248103 2578 generic.go:356] "Generic (PLEG): container finished" podID="52f44df7-1c9e-4e78-9c0a-b989bf1f70c7" containerID="f0845e6cd9c5566f2e07b5687a504c7dd95b373501f9c6a58c9df7f71902cf80" exitCode=0 Jul 23 12:10:28.248611 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:28.248165 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sdx4n" event={"ID":"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7","Type":"ContainerDied","Data":"f0845e6cd9c5566f2e07b5687a504c7dd95b373501f9c6a58c9df7f71902cf80"} Jul 23 12:10:29.023542 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:29.023279 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:29.023730 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:29.023590 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls\") pod \"dns-default-2wl6p\" (UID: \"f8597295-ae1a-42e6-ac3f-293915f8a69c\") " pod="openshift-dns/dns-default-2wl6p" Jul 23 12:10:29.023730 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:29.023442 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 12:10:29.023730 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:29.023639 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5999b54cbc-wg5fw: secret "image-registry-tls" not found Jul 23 12:10:29.023730 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:29.023713 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls podName:c3e44bef-488e-4d15-bd18-6cbbeec2b5eb nodeName:}" failed. No retries permitted until 2026-07-23 12:10:33.023687958 +0000 UTC m=+44.616222646 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls") pod "image-registry-5999b54cbc-wg5fw" (UID: "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb") : secret "image-registry-tls" not found Jul 23 12:10:29.023895 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:29.023726 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 12:10:29.023895 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:29.023784 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls podName:f8597295-ae1a-42e6-ac3f-293915f8a69c nodeName:}" failed. No retries permitted until 2026-07-23 12:10:33.023766059 +0000 UTC m=+44.616300740 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls") pod "dns-default-2wl6p" (UID: "f8597295-ae1a-42e6-ac3f-293915f8a69c") : secret "dns-default-metrics-tls" not found Jul 23 12:10:29.254819 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:29.254783 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sdx4n" event={"ID":"52f44df7-1c9e-4e78-9c0a-b989bf1f70c7","Type":"ContainerStarted","Data":"073922ca17c7c4a3011ccc4e21cb4dbfffa633b6774f098ad173a26822e55cc7"} Jul 23 12:10:29.282170 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:29.282063 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-sdx4n" podStartSLOduration=9.511309933 podStartE2EDuration="40.28204784s" totalImagesPullingTime="30.770737907s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:09:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:29.280324233 +0000 UTC m=+40.872858926" watchObservedRunningTime="2026-07-23 12:10:29.28204784 +0000 UTC m=+40.874582533" Jul 23 12:10:30.259537 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:30.259487 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ffzsb" event={"ID":"d13b8306-c63d-4eaa-92be-8ba79051dbd3","Type":"ContainerStarted","Data":"2ffd1b62a572be87a7c782138f9332676fac56daeffe4ebf6433fd6b11b117d9"} Jul 23 12:10:30.279616 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:30.279562 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ffzsb" podStartSLOduration=1.623050383 podStartE2EDuration="5.279547323s" totalImagesPullingTime="3.65649694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:10:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:30.278336542 +0000 UTC m=+41.870871237" watchObservedRunningTime="2026-07-23 12:10:30.279547323 +0000 UTC m=+41.872082050" Jul 23 12:10:33.052563 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:33.052523 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:33.052951 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:33.052579 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls\") pod \"dns-default-2wl6p\" (UID: \"f8597295-ae1a-42e6-ac3f-293915f8a69c\") " pod="openshift-dns/dns-default-2wl6p" Jul 23 12:10:33.052951 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:33.052688 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 12:10:33.052951 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:33.052715 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5999b54cbc-wg5fw: secret "image-registry-tls" not found Jul 23 12:10:33.052951 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:33.052774 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls podName:c3e44bef-488e-4d15-bd18-6cbbeec2b5eb nodeName:}" failed. No retries permitted until 2026-07-23 12:10:41.052760147 +0000 UTC m=+52.645294817 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls") pod "image-registry-5999b54cbc-wg5fw" (UID: "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb") : secret "image-registry-tls" not found Jul 23 12:10:33.052951 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:33.052698 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 12:10:33.052951 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:33.052830 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls podName:f8597295-ae1a-42e6-ac3f-293915f8a69c nodeName:}" failed. No retries permitted until 2026-07-23 12:10:41.052816205 +0000 UTC m=+52.645350877 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls") pod "dns-default-2wl6p" (UID: "f8597295-ae1a-42e6-ac3f-293915f8a69c") : secret "dns-default-metrics-tls" not found Jul 23 12:10:41.106583 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:41.106537 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:41.107095 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:41.106610 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls\") pod \"dns-default-2wl6p\" (UID: \"f8597295-ae1a-42e6-ac3f-293915f8a69c\") " pod="openshift-dns/dns-default-2wl6p" Jul 23 12:10:41.107095 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:41.106703 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 12:10:41.107095 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:41.106723 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5999b54cbc-wg5fw: secret "image-registry-tls" not found Jul 23 12:10:41.107095 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:41.106728 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 12:10:41.107095 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:41.106796 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls podName:f8597295-ae1a-42e6-ac3f-293915f8a69c nodeName:}" failed. No retries permitted until 2026-07-23 12:10:57.106773871 +0000 UTC m=+68.699308558 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls") pod "dns-default-2wl6p" (UID: "f8597295-ae1a-42e6-ac3f-293915f8a69c") : secret "dns-default-metrics-tls" not found Jul 23 12:10:41.107095 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:41.106816 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls podName:c3e44bef-488e-4d15-bd18-6cbbeec2b5eb nodeName:}" failed. No retries permitted until 2026-07-23 12:10:57.106807145 +0000 UTC m=+68.699341831 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls") pod "image-registry-5999b54cbc-wg5fw" (UID: "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb") : secret "image-registry-tls" not found Jul 23 12:10:43.300892 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.300853 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-8nqgf"] Jul 23 12:10:43.321721 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.321693 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-8nqgf"] Jul 23 12:10:43.321861 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.321794 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-8nqgf" Jul 23 12:10:43.324567 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.324537 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 23 12:10:43.324567 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.324550 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 23 12:10:43.324755 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.324573 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 23 12:10:43.324755 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.324658 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 23 12:10:43.325766 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.325751 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-ph869\"" Jul 23 12:10:43.329749 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.329731 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 23 12:10:43.423561 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.423521 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4eaee556-6e0b-46ae-9af7-da2785b3e34a-snapshots\") pod \"insights-operator-555d9d7885-8nqgf\" (UID: \"4eaee556-6e0b-46ae-9af7-da2785b3e34a\") " pod="openshift-insights/insights-operator-555d9d7885-8nqgf" Jul 23 12:10:43.423561 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.423557 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eaee556-6e0b-46ae-9af7-da2785b3e34a-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-8nqgf\" (UID: \"4eaee556-6e0b-46ae-9af7-da2785b3e34a\") " pod="openshift-insights/insights-operator-555d9d7885-8nqgf" Jul 23 12:10:43.423769 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.423591 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4eaee556-6e0b-46ae-9af7-da2785b3e34a-serving-cert\") pod \"insights-operator-555d9d7885-8nqgf\" (UID: \"4eaee556-6e0b-46ae-9af7-da2785b3e34a\") " pod="openshift-insights/insights-operator-555d9d7885-8nqgf" Jul 23 12:10:43.423769 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.423675 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6crm\" (UniqueName: \"kubernetes.io/projected/4eaee556-6e0b-46ae-9af7-da2785b3e34a-kube-api-access-p6crm\") pod \"insights-operator-555d9d7885-8nqgf\" (UID: \"4eaee556-6e0b-46ae-9af7-da2785b3e34a\") " pod="openshift-insights/insights-operator-555d9d7885-8nqgf" Jul 23 12:10:43.423769 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.423697 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4eaee556-6e0b-46ae-9af7-da2785b3e34a-tmp\") pod \"insights-operator-555d9d7885-8nqgf\" (UID: \"4eaee556-6e0b-46ae-9af7-da2785b3e34a\") " pod="openshift-insights/insights-operator-555d9d7885-8nqgf" Jul 23 12:10:43.423769 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.423723 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eaee556-6e0b-46ae-9af7-da2785b3e34a-service-ca-bundle\") pod \"insights-operator-555d9d7885-8nqgf\" (UID: \"4eaee556-6e0b-46ae-9af7-da2785b3e34a\") " pod="openshift-insights/insights-operator-555d9d7885-8nqgf" Jul 23 12:10:43.504754 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.504723 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-g68ch"] Jul 23 12:10:43.524159 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.524123 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4eaee556-6e0b-46ae-9af7-da2785b3e34a-snapshots\") pod \"insights-operator-555d9d7885-8nqgf\" (UID: \"4eaee556-6e0b-46ae-9af7-da2785b3e34a\") " pod="openshift-insights/insights-operator-555d9d7885-8nqgf" Jul 23 12:10:43.524350 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.524169 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eaee556-6e0b-46ae-9af7-da2785b3e34a-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-8nqgf\" (UID: \"4eaee556-6e0b-46ae-9af7-da2785b3e34a\") " pod="openshift-insights/insights-operator-555d9d7885-8nqgf" Jul 23 12:10:43.524350 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.524239 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4eaee556-6e0b-46ae-9af7-da2785b3e34a-serving-cert\") pod \"insights-operator-555d9d7885-8nqgf\" (UID: \"4eaee556-6e0b-46ae-9af7-da2785b3e34a\") " pod="openshift-insights/insights-operator-555d9d7885-8nqgf" Jul 23 12:10:43.524350 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.524297 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6crm\" (UniqueName: \"kubernetes.io/projected/4eaee556-6e0b-46ae-9af7-da2785b3e34a-kube-api-access-p6crm\") pod \"insights-operator-555d9d7885-8nqgf\" (UID: \"4eaee556-6e0b-46ae-9af7-da2785b3e34a\") " pod="openshift-insights/insights-operator-555d9d7885-8nqgf" Jul 23 12:10:43.524350 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.524331 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4eaee556-6e0b-46ae-9af7-da2785b3e34a-tmp\") pod \"insights-operator-555d9d7885-8nqgf\" (UID: \"4eaee556-6e0b-46ae-9af7-da2785b3e34a\") " pod="openshift-insights/insights-operator-555d9d7885-8nqgf" Jul 23 12:10:43.524562 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.524376 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eaee556-6e0b-46ae-9af7-da2785b3e34a-service-ca-bundle\") pod \"insights-operator-555d9d7885-8nqgf\" (UID: \"4eaee556-6e0b-46ae-9af7-da2785b3e34a\") " pod="openshift-insights/insights-operator-555d9d7885-8nqgf" Jul 23 12:10:43.524934 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.524909 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4eaee556-6e0b-46ae-9af7-da2785b3e34a-tmp\") pod \"insights-operator-555d9d7885-8nqgf\" (UID: \"4eaee556-6e0b-46ae-9af7-da2785b3e34a\") " pod="openshift-insights/insights-operator-555d9d7885-8nqgf" Jul 23 12:10:43.525043 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.524994 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4eaee556-6e0b-46ae-9af7-da2785b3e34a-snapshots\") pod \"insights-operator-555d9d7885-8nqgf\" (UID: \"4eaee556-6e0b-46ae-9af7-da2785b3e34a\") " pod="openshift-insights/insights-operator-555d9d7885-8nqgf" Jul 23 12:10:43.525786 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.525133 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eaee556-6e0b-46ae-9af7-da2785b3e34a-service-ca-bundle\") pod \"insights-operator-555d9d7885-8nqgf\" (UID: \"4eaee556-6e0b-46ae-9af7-da2785b3e34a\") " pod="openshift-insights/insights-operator-555d9d7885-8nqgf" Jul 23 12:10:43.525786 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.525694 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eaee556-6e0b-46ae-9af7-da2785b3e34a-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-8nqgf\" (UID: \"4eaee556-6e0b-46ae-9af7-da2785b3e34a\") " pod="openshift-insights/insights-operator-555d9d7885-8nqgf" Jul 23 12:10:43.528442 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.528419 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fwbjz"] Jul 23 12:10:43.528653 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.528544 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-g68ch" Jul 23 12:10:43.529478 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.529457 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4eaee556-6e0b-46ae-9af7-da2785b3e34a-serving-cert\") pod \"insights-operator-555d9d7885-8nqgf\" (UID: \"4eaee556-6e0b-46ae-9af7-da2785b3e34a\") " pod="openshift-insights/insights-operator-555d9d7885-8nqgf" Jul 23 12:10:43.531446 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.531431 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 23 12:10:43.531533 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.531467 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 23 12:10:43.532017 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.532002 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-kbrrc\"" Jul 23 12:10:43.540181 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.540161 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6crm\" (UniqueName: \"kubernetes.io/projected/4eaee556-6e0b-46ae-9af7-da2785b3e34a-kube-api-access-p6crm\") pod \"insights-operator-555d9d7885-8nqgf\" (UID: \"4eaee556-6e0b-46ae-9af7-da2785b3e34a\") " pod="openshift-insights/insights-operator-555d9d7885-8nqgf" Jul 23 12:10:43.548461 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.548440 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cns49"] Jul 23 12:10:43.548585 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.548569 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fwbjz" Jul 23 12:10:43.551221 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.551154 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 23 12:10:43.551785 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.551764 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 23 12:10:43.552049 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.552034 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 23 12:10:43.552127 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.552052 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 23 12:10:43.552127 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.552097 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-ddk7w\"" Jul 23 12:10:43.574677 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.574649 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-6btjx"] Jul 23 12:10:43.574857 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.574784 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cns49" Jul 23 12:10:43.577561 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.577530 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 23 12:10:43.577561 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.577538 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 23 12:10:43.577727 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.577630 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-2b2gc\"" Jul 23 12:10:43.577799 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.577781 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 23 12:10:43.578148 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.578135 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 23 12:10:43.594914 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.594880 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-g68ch"] Jul 23 12:10:43.595058 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.594926 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fwbjz"] Jul 23 12:10:43.595058 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.594949 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-6btjx"] Jul 23 12:10:43.595058 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.594967 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cns49"] Jul 23 12:10:43.595058 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.595005 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-6btjx" Jul 23 12:10:43.597409 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.597391 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-24wnb\"" Jul 23 12:10:43.625028 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.624993 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrg42\" (UniqueName: \"kubernetes.io/projected/31dc0cb0-5436-407b-8408-fcf7a127e670-kube-api-access-lrg42\") pod \"network-check-source-7bc5b6bf85-6btjx\" (UID: \"31dc0cb0-5436-407b-8408-fcf7a127e670\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-6btjx" Jul 23 12:10:43.625028 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.625029 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf-serving-cert\") pod \"service-ca-operator-5bcbc94c78-fwbjz\" (UID: \"5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fwbjz" Jul 23 12:10:43.625272 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.625061 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fafa146f-b22a-49a7-b7c6-263e59d7d9ab-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-cns49\" (UID: \"fafa146f-b22a-49a7-b7c6-263e59d7d9ab\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cns49" Jul 23 12:10:43.625272 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.625088 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fkb8\" (UniqueName: \"kubernetes.io/projected/034d97e3-9d7e-4957-8235-25b1eb22eb85-kube-api-access-9fkb8\") pod \"volume-data-source-validator-78cf4c6f6-g68ch\" (UID: \"034d97e3-9d7e-4957-8235-25b1eb22eb85\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-g68ch" Jul 23 12:10:43.625272 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.625130 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf-config\") pod \"service-ca-operator-5bcbc94c78-fwbjz\" (UID: \"5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fwbjz" Jul 23 12:10:43.625272 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.625158 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkbhf\" (UniqueName: \"kubernetes.io/projected/5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf-kube-api-access-mkbhf\") pod \"service-ca-operator-5bcbc94c78-fwbjz\" (UID: \"5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fwbjz" Jul 23 12:10:43.625272 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.625181 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fafa146f-b22a-49a7-b7c6-263e59d7d9ab-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-cns49\" (UID: \"fafa146f-b22a-49a7-b7c6-263e59d7d9ab\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cns49" Jul 23 12:10:43.625437 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.625291 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7r6t\" (UniqueName: \"kubernetes.io/projected/fafa146f-b22a-49a7-b7c6-263e59d7d9ab-kube-api-access-r7r6t\") pod \"kube-storage-version-migrator-operator-7658d9b749-cns49\" (UID: \"fafa146f-b22a-49a7-b7c6-263e59d7d9ab\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cns49" Jul 23 12:10:43.631047 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.631027 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-8nqgf" Jul 23 12:10:43.725646 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.725616 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrg42\" (UniqueName: \"kubernetes.io/projected/31dc0cb0-5436-407b-8408-fcf7a127e670-kube-api-access-lrg42\") pod \"network-check-source-7bc5b6bf85-6btjx\" (UID: \"31dc0cb0-5436-407b-8408-fcf7a127e670\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-6btjx" Jul 23 12:10:43.725783 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.725657 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf-serving-cert\") pod \"service-ca-operator-5bcbc94c78-fwbjz\" (UID: \"5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fwbjz" Jul 23 12:10:43.725783 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.725679 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fafa146f-b22a-49a7-b7c6-263e59d7d9ab-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-cns49\" (UID: \"fafa146f-b22a-49a7-b7c6-263e59d7d9ab\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cns49" Jul 23 12:10:43.725783 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.725698 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fkb8\" (UniqueName: \"kubernetes.io/projected/034d97e3-9d7e-4957-8235-25b1eb22eb85-kube-api-access-9fkb8\") pod \"volume-data-source-validator-78cf4c6f6-g68ch\" (UID: \"034d97e3-9d7e-4957-8235-25b1eb22eb85\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-g68ch" Jul 23 12:10:43.725783 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.725727 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf-config\") pod \"service-ca-operator-5bcbc94c78-fwbjz\" (UID: \"5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fwbjz" Jul 23 12:10:43.725994 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.725804 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkbhf\" (UniqueName: \"kubernetes.io/projected/5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf-kube-api-access-mkbhf\") pod \"service-ca-operator-5bcbc94c78-fwbjz\" (UID: \"5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fwbjz" Jul 23 12:10:43.725994 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.725848 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fafa146f-b22a-49a7-b7c6-263e59d7d9ab-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-cns49\" (UID: \"fafa146f-b22a-49a7-b7c6-263e59d7d9ab\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cns49" Jul 23 12:10:43.725994 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.725955 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7r6t\" (UniqueName: \"kubernetes.io/projected/fafa146f-b22a-49a7-b7c6-263e59d7d9ab-kube-api-access-r7r6t\") pod \"kube-storage-version-migrator-operator-7658d9b749-cns49\" (UID: \"fafa146f-b22a-49a7-b7c6-263e59d7d9ab\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cns49" Jul 23 12:10:43.726451 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.726340 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf-config\") pod \"service-ca-operator-5bcbc94c78-fwbjz\" (UID: \"5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fwbjz" Jul 23 12:10:43.726748 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.726727 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fafa146f-b22a-49a7-b7c6-263e59d7d9ab-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-cns49\" (UID: \"fafa146f-b22a-49a7-b7c6-263e59d7d9ab\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cns49" Jul 23 12:10:43.728520 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.728497 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fafa146f-b22a-49a7-b7c6-263e59d7d9ab-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-cns49\" (UID: \"fafa146f-b22a-49a7-b7c6-263e59d7d9ab\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cns49" Jul 23 12:10:43.728599 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.728539 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf-serving-cert\") pod \"service-ca-operator-5bcbc94c78-fwbjz\" (UID: \"5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fwbjz" Jul 23 12:10:43.737381 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.736917 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7r6t\" (UniqueName: \"kubernetes.io/projected/fafa146f-b22a-49a7-b7c6-263e59d7d9ab-kube-api-access-r7r6t\") pod \"kube-storage-version-migrator-operator-7658d9b749-cns49\" (UID: \"fafa146f-b22a-49a7-b7c6-263e59d7d9ab\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cns49" Jul 23 12:10:43.737506 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.737429 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fkb8\" (UniqueName: \"kubernetes.io/projected/034d97e3-9d7e-4957-8235-25b1eb22eb85-kube-api-access-9fkb8\") pod \"volume-data-source-validator-78cf4c6f6-g68ch\" (UID: \"034d97e3-9d7e-4957-8235-25b1eb22eb85\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-g68ch" Jul 23 12:10:43.738119 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.738092 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrg42\" (UniqueName: \"kubernetes.io/projected/31dc0cb0-5436-407b-8408-fcf7a127e670-kube-api-access-lrg42\") pod \"network-check-source-7bc5b6bf85-6btjx\" (UID: \"31dc0cb0-5436-407b-8408-fcf7a127e670\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-6btjx" Jul 23 12:10:43.739522 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.739501 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkbhf\" (UniqueName: \"kubernetes.io/projected/5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf-kube-api-access-mkbhf\") pod \"service-ca-operator-5bcbc94c78-fwbjz\" (UID: \"5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fwbjz" Jul 23 12:10:43.778334 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.778304 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-8nqgf"] Jul 23 12:10:43.781391 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:10:43.781361 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eaee556_6e0b_46ae_9af7_da2785b3e34a.slice/crio-75f016b22d78309b293ebbe46aceafe429d00efaedb42efa929f13ed16bc6167 WatchSource:0}: Error finding container 75f016b22d78309b293ebbe46aceafe429d00efaedb42efa929f13ed16bc6167: Status 404 returned error can't find the container with id 75f016b22d78309b293ebbe46aceafe429d00efaedb42efa929f13ed16bc6167 Jul 23 12:10:43.841914 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.841826 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-g68ch" Jul 23 12:10:43.857046 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.857008 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fwbjz" Jul 23 12:10:43.882858 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.882834 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cns49" Jul 23 12:10:43.903653 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:43.903626 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-6btjx" Jul 23 12:10:44.032208 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:44.032161 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-g68ch"] Jul 23 12:10:44.035351 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:10:44.035317 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod034d97e3_9d7e_4957_8235_25b1eb22eb85.slice/crio-1e429f4c8a20837eb89afb53ff04aafbc31322f9974c6d6225cfe290e0449cb6 WatchSource:0}: Error finding container 1e429f4c8a20837eb89afb53ff04aafbc31322f9974c6d6225cfe290e0449cb6: Status 404 returned error can't find the container with id 1e429f4c8a20837eb89afb53ff04aafbc31322f9974c6d6225cfe290e0449cb6 Jul 23 12:10:44.053495 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:44.053461 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fwbjz"] Jul 23 12:10:44.057695 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:10:44.057661 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c791e2d_cdef_4dd1_8fb9_e0dcc8243bbf.slice/crio-ae0b0292642493cd1aaa2ee39c93cf5c128a9cb7fa03c53a8c8b8dc450acbd5a WatchSource:0}: Error finding container ae0b0292642493cd1aaa2ee39c93cf5c128a9cb7fa03c53a8c8b8dc450acbd5a: Status 404 returned error can't find the container with id ae0b0292642493cd1aaa2ee39c93cf5c128a9cb7fa03c53a8c8b8dc450acbd5a Jul 23 12:10:44.074287 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:44.074263 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cns49"] Jul 23 12:10:44.077548 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:10:44.077526 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfafa146f_b22a_49a7_b7c6_263e59d7d9ab.slice/crio-219f86ad143f4a6f753e400ce1cf4465ff2c5e475b53cfe00093daf69a794844 WatchSource:0}: Error finding container 219f86ad143f4a6f753e400ce1cf4465ff2c5e475b53cfe00093daf69a794844: Status 404 returned error can't find the container with id 219f86ad143f4a6f753e400ce1cf4465ff2c5e475b53cfe00093daf69a794844 Jul 23 12:10:44.088932 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:44.088910 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-6btjx"] Jul 23 12:10:44.093145 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:10:44.093092 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31dc0cb0_5436_407b_8408_fcf7a127e670.slice/crio-b52f446f5595c4e25a076691b1948fdb70ad0a0a5335eb645f0bccafa600cf26 WatchSource:0}: Error finding container b52f446f5595c4e25a076691b1948fdb70ad0a0a5335eb645f0bccafa600cf26: Status 404 returned error can't find the container with id b52f446f5595c4e25a076691b1948fdb70ad0a0a5335eb645f0bccafa600cf26 Jul 23 12:10:44.292398 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:44.292364 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fwbjz" event={"ID":"5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf","Type":"ContainerStarted","Data":"ae0b0292642493cd1aaa2ee39c93cf5c128a9cb7fa03c53a8c8b8dc450acbd5a"} Jul 23 12:10:44.293448 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:44.293428 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-g68ch" event={"ID":"034d97e3-9d7e-4957-8235-25b1eb22eb85","Type":"ContainerStarted","Data":"1e429f4c8a20837eb89afb53ff04aafbc31322f9974c6d6225cfe290e0449cb6"} Jul 23 12:10:44.294356 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:44.294334 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cns49" event={"ID":"fafa146f-b22a-49a7-b7c6-263e59d7d9ab","Type":"ContainerStarted","Data":"219f86ad143f4a6f753e400ce1cf4465ff2c5e475b53cfe00093daf69a794844"} Jul 23 12:10:44.295311 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:44.295294 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-6btjx" event={"ID":"31dc0cb0-5436-407b-8408-fcf7a127e670","Type":"ContainerStarted","Data":"b52f446f5595c4e25a076691b1948fdb70ad0a0a5335eb645f0bccafa600cf26"} Jul 23 12:10:44.296179 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:44.296163 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-8nqgf" event={"ID":"4eaee556-6e0b-46ae-9af7-da2785b3e34a","Type":"ContainerStarted","Data":"75f016b22d78309b293ebbe46aceafe429d00efaedb42efa929f13ed16bc6167"} Jul 23 12:10:46.304118 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:46.304052 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-g68ch" event={"ID":"034d97e3-9d7e-4957-8235-25b1eb22eb85","Type":"ContainerStarted","Data":"456190bcf2d941d03444bc6d919a93af946a3380e4a19bb3861dcdc2ff59dab0"} Jul 23 12:10:46.322342 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:46.322284 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-g68ch" podStartSLOduration=1.950982646 podStartE2EDuration="3.322262711s" totalImagesPullingTime="1.371280065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:10:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:46.321072976 +0000 UTC m=+57.913607670" watchObservedRunningTime="2026-07-23 12:10:46.322262711 +0000 UTC m=+57.914797403" Jul 23 12:10:48.314075 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:48.314029 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cns49" event={"ID":"fafa146f-b22a-49a7-b7c6-263e59d7d9ab","Type":"ContainerStarted","Data":"1ef3107853cac33d1d4134fe0568facaf182ceac5f1e91593eb9953dbf7bd4ab"} Jul 23 12:10:48.316278 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:48.316224 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-6btjx" event={"ID":"31dc0cb0-5436-407b-8408-fcf7a127e670","Type":"ContainerStarted","Data":"23d4fed2b3206ee5438a2922bdb99071e7b97d5c6ba5cc50821127dfed8dad49"} Jul 23 12:10:48.318082 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:48.317866 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-8nqgf" event={"ID":"4eaee556-6e0b-46ae-9af7-da2785b3e34a","Type":"ContainerStarted","Data":"59134bd7d25d1e170291be25e0ba428fff0e2e2e7fdad052b239604521a7ab74"} Jul 23 12:10:48.319476 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:48.319449 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fwbjz" event={"ID":"5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf","Type":"ContainerStarted","Data":"791d7d63f4635839a4267442252167a563b57723330d3a77a92b525a416e6cbd"} Jul 23 12:10:48.334967 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:48.334924 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cns49" podStartSLOduration=1.312020858 podStartE2EDuration="5.334912647s" totalImagesPullingTime="4.022891789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:10:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:48.334604265 +0000 UTC m=+59.927138970" watchObservedRunningTime="2026-07-23 12:10:48.334912647 +0000 UTC m=+59.927447339" Jul 23 12:10:48.353326 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:48.353280 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fwbjz" podStartSLOduration=1.306112709 podStartE2EDuration="5.353261386s" totalImagesPullingTime="4.047148677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:10:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:48.351786203 +0000 UTC m=+59.944320897" watchObservedRunningTime="2026-07-23 12:10:48.353261386 +0000 UTC m=+59.945796081" Jul 23 12:10:48.377399 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:48.377346 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-8nqgf" podStartSLOduration=1.060092136 podStartE2EDuration="5.377329914s" totalImagesPullingTime="4.317237778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:10:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:48.376518004 +0000 UTC m=+59.969052698" watchObservedRunningTime="2026-07-23 12:10:48.377329914 +0000 UTC m=+59.969864608" Jul 23 12:10:49.369013 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.368946 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-6btjx" podStartSLOduration=2.355267179 podStartE2EDuration="6.368924186s" totalImagesPullingTime="4.013657007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:10:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:48.395099283 +0000 UTC m=+59.987633980" watchObservedRunningTime="2026-07-23 12:10:49.368924186 +0000 UTC m=+60.961458879" Jul 23 12:10:49.369790 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.369766 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7z6nv"] Jul 23 12:10:49.373212 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.373165 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:49.377146 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.377118 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 23 12:10:49.377456 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.377436 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-l7bff\"" Jul 23 12:10:49.377636 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.377617 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 23 12:10:49.392509 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.392482 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7z6nv"] Jul 23 12:10:49.479385 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.479363 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/09f3adc9-0ded-4260-bedb-cf97199c0a39-crio-socket\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:49.479530 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.479396 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:49.479530 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.479469 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09f3adc9-0ded-4260-bedb-cf97199c0a39-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:49.479530 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.479517 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/09f3adc9-0ded-4260-bedb-cf97199c0a39-data-volume\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:49.479659 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.479568 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t58tt\" (UniqueName: \"kubernetes.io/projected/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-api-access-t58tt\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:49.580763 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.580726 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/09f3adc9-0ded-4260-bedb-cf97199c0a39-crio-socket\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:49.580961 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.580776 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:49.580961 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.580828 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09f3adc9-0ded-4260-bedb-cf97199c0a39-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:49.580961 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.580860 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/09f3adc9-0ded-4260-bedb-cf97199c0a39-data-volume\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:49.580961 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.580867 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/09f3adc9-0ded-4260-bedb-cf97199c0a39-crio-socket\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:49.580961 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.580903 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t58tt\" (UniqueName: \"kubernetes.io/projected/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-api-access-t58tt\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:49.580961 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:49.580933 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 12:10:49.581279 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:49.580986 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 12:10:49.581279 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:49.581011 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-rbac-proxy-cm podName:09f3adc9-0ded-4260-bedb-cf97199c0a39 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:50.080988729 +0000 UTC m=+61.673523406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7z6nv" (UID: "09f3adc9-0ded-4260-bedb-cf97199c0a39") : configmap "kube-rbac-proxy" not found Jul 23 12:10:49.581279 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:49.581044 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09f3adc9-0ded-4260-bedb-cf97199c0a39-insights-runtime-extractor-tls podName:09f3adc9-0ded-4260-bedb-cf97199c0a39 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:50.081027784 +0000 UTC m=+61.673562458 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09f3adc9-0ded-4260-bedb-cf97199c0a39-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7z6nv" (UID: "09f3adc9-0ded-4260-bedb-cf97199c0a39") : secret "insights-runtime-extractor-tls" not found Jul 23 12:10:49.581279 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.581149 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/09f3adc9-0ded-4260-bedb-cf97199c0a39-data-volume\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:49.596759 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.596727 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t58tt\" (UniqueName: \"kubernetes.io/projected/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-api-access-t58tt\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:49.604809 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.604786 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-lh8jx"] Jul 23 12:10:49.607228 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.607210 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-lh8jx" Jul 23 12:10:49.609774 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.609753 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-5rdjt\"" Jul 23 12:10:49.609895 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.609813 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 23 12:10:49.610102 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.610084 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 23 12:10:49.618748 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.618727 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-lh8jx"] Jul 23 12:10:49.681631 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.681562 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhhwd\" (UniqueName: \"kubernetes.io/projected/3020a32a-dfd8-4f43-94ea-bb675c26f0fe-kube-api-access-jhhwd\") pod \"migrator-7cc9c7ffd7-lh8jx\" (UID: \"3020a32a-dfd8-4f43-94ea-bb675c26f0fe\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-lh8jx" Jul 23 12:10:49.782094 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.782063 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhhwd\" (UniqueName: \"kubernetes.io/projected/3020a32a-dfd8-4f43-94ea-bb675c26f0fe-kube-api-access-jhhwd\") pod \"migrator-7cc9c7ffd7-lh8jx\" (UID: \"3020a32a-dfd8-4f43-94ea-bb675c26f0fe\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-lh8jx" Jul 23 12:10:49.808097 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.808064 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhhwd\" (UniqueName: \"kubernetes.io/projected/3020a32a-dfd8-4f43-94ea-bb675c26f0fe-kube-api-access-jhhwd\") pod \"migrator-7cc9c7ffd7-lh8jx\" (UID: \"3020a32a-dfd8-4f43-94ea-bb675c26f0fe\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-lh8jx" Jul 23 12:10:49.917713 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:49.917677 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-lh8jx" Jul 23 12:10:50.047012 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:50.046983 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-lh8jx"] Jul 23 12:10:50.050426 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:10:50.050395 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3020a32a_dfd8_4f43_94ea_bb675c26f0fe.slice/crio-e60344c2883fcce420ab2480496267bd0fd150ac80fb9810345298a0b75ad119 WatchSource:0}: Error finding container e60344c2883fcce420ab2480496267bd0fd150ac80fb9810345298a0b75ad119: Status 404 returned error can't find the container with id e60344c2883fcce420ab2480496267bd0fd150ac80fb9810345298a0b75ad119 Jul 23 12:10:50.084435 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:50.084412 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09f3adc9-0ded-4260-bedb-cf97199c0a39-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:50.084540 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:50.084527 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:50.084633 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:50.084618 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 12:10:50.084690 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:50.084666 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09f3adc9-0ded-4260-bedb-cf97199c0a39-insights-runtime-extractor-tls podName:09f3adc9-0ded-4260-bedb-cf97199c0a39 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:51.084652276 +0000 UTC m=+62.677186950 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09f3adc9-0ded-4260-bedb-cf97199c0a39-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7z6nv" (UID: "09f3adc9-0ded-4260-bedb-cf97199c0a39") : secret "insights-runtime-extractor-tls" not found Jul 23 12:10:50.084755 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:50.084623 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 12:10:50.084802 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:50.084765 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-rbac-proxy-cm podName:09f3adc9-0ded-4260-bedb-cf97199c0a39 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:51.084746506 +0000 UTC m=+62.677281191 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7z6nv" (UID: "09f3adc9-0ded-4260-bedb-cf97199c0a39") : configmap "kube-rbac-proxy" not found Jul 23 12:10:50.326793 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:50.326713 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-lh8jx" event={"ID":"3020a32a-dfd8-4f43-94ea-bb675c26f0fe","Type":"ContainerStarted","Data":"e60344c2883fcce420ab2480496267bd0fd150ac80fb9810345298a0b75ad119"} Jul 23 12:10:51.093281 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:51.093243 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:51.093728 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:51.093307 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09f3adc9-0ded-4260-bedb-cf97199c0a39-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:51.093728 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:51.093376 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 12:10:51.093728 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:51.093439 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 12:10:51.093728 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:51.093449 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-rbac-proxy-cm podName:09f3adc9-0ded-4260-bedb-cf97199c0a39 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:53.093433122 +0000 UTC m=+64.685967793 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7z6nv" (UID: "09f3adc9-0ded-4260-bedb-cf97199c0a39") : configmap "kube-rbac-proxy" not found Jul 23 12:10:51.093728 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:51.093510 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09f3adc9-0ded-4260-bedb-cf97199c0a39-insights-runtime-extractor-tls podName:09f3adc9-0ded-4260-bedb-cf97199c0a39 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:53.09349373 +0000 UTC m=+64.686028401 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09f3adc9-0ded-4260-bedb-cf97199c0a39-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7z6nv" (UID: "09f3adc9-0ded-4260-bedb-cf97199c0a39") : secret "insights-runtime-extractor-tls" not found Jul 23 12:10:51.172397 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:51.172360 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rj4l2_d3c0c881-554f-4572-bdaa-c0ff060c50ec/dns-node-resolver/0.log" Jul 23 12:10:51.233595 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:51.233564 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jsxrj" Jul 23 12:10:52.172410 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.172386 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zdc7h_374f6cfe-ca3e-424f-9875-3040a3e78447/node-ca/0.log" Jul 23 12:10:52.336102 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.336066 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-lh8jx" event={"ID":"3020a32a-dfd8-4f43-94ea-bb675c26f0fe","Type":"ContainerStarted","Data":"bc13eeebcbf6de73df61deefcf761768e927fc8907dad29ab8b87ae8926d9040"} Jul 23 12:10:52.336305 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.336116 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-lh8jx" event={"ID":"3020a32a-dfd8-4f43-94ea-bb675c26f0fe","Type":"ContainerStarted","Data":"291623899fc8e2db374cfc3b141bbe6a37dee584a904cb33c690c9ba5308bb57"} Jul 23 12:10:52.354501 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.354453 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-lh8jx" podStartSLOduration=1.7198644889999999 podStartE2EDuration="3.354438743s" totalImagesPullingTime="1.634574254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:52.353682503 +0000 UTC m=+63.946217197" watchObservedRunningTime="2026-07-23 12:10:52.354438743 +0000 UTC m=+63.946973436" Jul 23 12:10:52.361182 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.361158 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-l6nwh"] Jul 23 12:10:52.363555 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.363539 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-l6nwh" Jul 23 12:10:52.365859 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.365835 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 23 12:10:52.366878 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.366862 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 23 12:10:52.368099 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.368080 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-cp46k\"" Jul 23 12:10:52.368179 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.368145 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 23 12:10:52.368179 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.368168 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 23 12:10:52.374972 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.374952 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-l6nwh"] Jul 23 12:10:52.402030 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.402005 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qkd7\" (UniqueName: \"kubernetes.io/projected/1ba17eed-1f92-440d-a18a-245da889a0de-kube-api-access-7qkd7\") pod \"service-ca-58598695f7-l6nwh\" (UID: \"1ba17eed-1f92-440d-a18a-245da889a0de\") " pod="openshift-service-ca/service-ca-58598695f7-l6nwh" Jul 23 12:10:52.402155 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.402036 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1ba17eed-1f92-440d-a18a-245da889a0de-signing-key\") pod \"service-ca-58598695f7-l6nwh\" (UID: \"1ba17eed-1f92-440d-a18a-245da889a0de\") " pod="openshift-service-ca/service-ca-58598695f7-l6nwh" Jul 23 12:10:52.402254 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.402235 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1ba17eed-1f92-440d-a18a-245da889a0de-signing-cabundle\") pod \"service-ca-58598695f7-l6nwh\" (UID: \"1ba17eed-1f92-440d-a18a-245da889a0de\") " pod="openshift-service-ca/service-ca-58598695f7-l6nwh" Jul 23 12:10:52.503400 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.503310 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qkd7\" (UniqueName: \"kubernetes.io/projected/1ba17eed-1f92-440d-a18a-245da889a0de-kube-api-access-7qkd7\") pod \"service-ca-58598695f7-l6nwh\" (UID: \"1ba17eed-1f92-440d-a18a-245da889a0de\") " pod="openshift-service-ca/service-ca-58598695f7-l6nwh" Jul 23 12:10:52.503400 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.503351 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1ba17eed-1f92-440d-a18a-245da889a0de-signing-key\") pod \"service-ca-58598695f7-l6nwh\" (UID: \"1ba17eed-1f92-440d-a18a-245da889a0de\") " pod="openshift-service-ca/service-ca-58598695f7-l6nwh" Jul 23 12:10:52.503637 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.503502 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1ba17eed-1f92-440d-a18a-245da889a0de-signing-cabundle\") pod \"service-ca-58598695f7-l6nwh\" (UID: \"1ba17eed-1f92-440d-a18a-245da889a0de\") " pod="openshift-service-ca/service-ca-58598695f7-l6nwh" Jul 23 12:10:52.504110 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.504086 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1ba17eed-1f92-440d-a18a-245da889a0de-signing-cabundle\") pod \"service-ca-58598695f7-l6nwh\" (UID: \"1ba17eed-1f92-440d-a18a-245da889a0de\") " pod="openshift-service-ca/service-ca-58598695f7-l6nwh" Jul 23 12:10:52.505731 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.505711 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1ba17eed-1f92-440d-a18a-245da889a0de-signing-key\") pod \"service-ca-58598695f7-l6nwh\" (UID: \"1ba17eed-1f92-440d-a18a-245da889a0de\") " pod="openshift-service-ca/service-ca-58598695f7-l6nwh" Jul 23 12:10:52.514716 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.514697 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qkd7\" (UniqueName: \"kubernetes.io/projected/1ba17eed-1f92-440d-a18a-245da889a0de-kube-api-access-7qkd7\") pod \"service-ca-58598695f7-l6nwh\" (UID: \"1ba17eed-1f92-440d-a18a-245da889a0de\") " pod="openshift-service-ca/service-ca-58598695f7-l6nwh" Jul 23 12:10:52.672301 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.672263 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-l6nwh" Jul 23 12:10:52.772153 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.772084 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ffzsb_d13b8306-c63d-4eaa-92be-8ba79051dbd3/serve-healthcheck-canary/0.log" Jul 23 12:10:52.792427 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:52.792396 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-l6nwh"] Jul 23 12:10:52.796824 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:10:52.796793 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ba17eed_1f92_440d_a18a_245da889a0de.slice/crio-01230cab6bc448c2589560302677c886d60a71470b11232d8afb5c572b570e7f WatchSource:0}: Error finding container 01230cab6bc448c2589560302677c886d60a71470b11232d8afb5c572b570e7f: Status 404 returned error can't find the container with id 01230cab6bc448c2589560302677c886d60a71470b11232d8afb5c572b570e7f Jul 23 12:10:53.108876 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:53.108833 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09f3adc9-0ded-4260-bedb-cf97199c0a39-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:53.109062 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:53.108987 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:53.109062 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:53.109006 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 12:10:53.109062 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:53.109058 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 12:10:53.109258 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:53.109091 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09f3adc9-0ded-4260-bedb-cf97199c0a39-insights-runtime-extractor-tls podName:09f3adc9-0ded-4260-bedb-cf97199c0a39 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:57.109070542 +0000 UTC m=+68.701605224 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09f3adc9-0ded-4260-bedb-cf97199c0a39-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7z6nv" (UID: "09f3adc9-0ded-4260-bedb-cf97199c0a39") : secret "insights-runtime-extractor-tls" not found Jul 23 12:10:53.109258 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:53.109113 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-rbac-proxy-cm podName:09f3adc9-0ded-4260-bedb-cf97199c0a39 nodeName:}" failed. No retries permitted until 2026-07-23 12:10:57.109103035 +0000 UTC m=+68.701637705 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7z6nv" (UID: "09f3adc9-0ded-4260-bedb-cf97199c0a39") : configmap "kube-rbac-proxy" not found Jul 23 12:10:53.345984 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:53.345941 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-l6nwh" event={"ID":"1ba17eed-1f92-440d-a18a-245da889a0de","Type":"ContainerStarted","Data":"d58f6c8f76076f273bfd2a60f972829bec462999cde8ae2c6417fda5f25ebb73"} Jul 23 12:10:53.346417 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:53.346030 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-l6nwh" event={"ID":"1ba17eed-1f92-440d-a18a-245da889a0de","Type":"ContainerStarted","Data":"01230cab6bc448c2589560302677c886d60a71470b11232d8afb5c572b570e7f"} Jul 23 12:10:53.366259 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:53.366140 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-l6nwh" podStartSLOduration=1.366123141 podStartE2EDuration="1.366123141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:10:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:53.365301446 +0000 UTC m=+64.957836142" watchObservedRunningTime="2026-07-23 12:10:53.366123141 +0000 UTC m=+64.958657890" Jul 23 12:10:54.623859 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:54.623812 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret\") pod \"global-pull-secret-syncer-tcgfq\" (UID: \"88f19386-8325-4469-9859-cd876612dcd6\") " pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:10:54.626726 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:54.626702 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 12:10:54.636987 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:54.636960 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/88f19386-8325-4469-9859-cd876612dcd6-original-pull-secret\") pod \"global-pull-secret-syncer-tcgfq\" (UID: \"88f19386-8325-4469-9859-cd876612dcd6\") " pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:10:54.725119 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:54.725084 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs\") pod \"network-metrics-daemon-cbm6t\" (UID: \"232bfbf8-288e-4679-8bee-4c5179866c00\") " pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:10:54.727804 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:54.727783 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 12:10:54.736221 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:54.736185 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 23 12:10:54.736340 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:54.736270 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs podName:232bfbf8-288e-4679-8bee-4c5179866c00 nodeName:}" failed. No retries permitted until 2026-07-23 12:11:58.736251126 +0000 UTC m=+130.328785814 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs") pod "network-metrics-daemon-cbm6t" (UID: "232bfbf8-288e-4679-8bee-4c5179866c00") : secret "metrics-daemon-secret" not found Jul 23 12:10:54.752910 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:54.752876 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcgfq" Jul 23 12:10:54.826225 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:54.826168 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2rbr\" (UniqueName: \"kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr\") pod \"network-check-target-tjdx4\" (UID: \"6b308275-101d-4a54-86d4-fc63739354cb\") " pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:10:54.830933 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:54.830904 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2rbr\" (UniqueName: \"kubernetes.io/projected/6b308275-101d-4a54-86d4-fc63739354cb-kube-api-access-m2rbr\") pod \"network-check-target-tjdx4\" (UID: \"6b308275-101d-4a54-86d4-fc63739354cb\") " pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:10:54.908147 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:54.908048 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tcgfq"] Jul 23 12:10:54.912796 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:10:54.912740 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88f19386_8325_4469_9859_cd876612dcd6.slice/crio-39ecbab88609d2550d937bc4e0282b14d0a1de39ffd7ce11943f97845e4554bc WatchSource:0}: Error finding container 39ecbab88609d2550d937bc4e0282b14d0a1de39ffd7ce11943f97845e4554bc: Status 404 returned error can't find the container with id 39ecbab88609d2550d937bc4e0282b14d0a1de39ffd7ce11943f97845e4554bc Jul 23 12:10:55.049837 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:55.049800 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pbg55\"" Jul 23 12:10:55.057143 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:55.057116 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:10:55.211054 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:55.210971 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tjdx4"] Jul 23 12:10:55.214612 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:10:55.214580 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b308275_101d_4a54_86d4_fc63739354cb.slice/crio-8fa776885fa5442440060c5b4801cb7740be6b0d16f2c60ea289f2bbbfb88f45 WatchSource:0}: Error finding container 8fa776885fa5442440060c5b4801cb7740be6b0d16f2c60ea289f2bbbfb88f45: Status 404 returned error can't find the container with id 8fa776885fa5442440060c5b4801cb7740be6b0d16f2c60ea289f2bbbfb88f45 Jul 23 12:10:55.353498 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:55.353467 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tjdx4" event={"ID":"6b308275-101d-4a54-86d4-fc63739354cb","Type":"ContainerStarted","Data":"0ef59f9f2beb9b5d6f4707a8977e904e129cbafceff40079b5e98384b6f4df4f"} Jul 23 12:10:55.353701 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:55.353515 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tjdx4" event={"ID":"6b308275-101d-4a54-86d4-fc63739354cb","Type":"ContainerStarted","Data":"8fa776885fa5442440060c5b4801cb7740be6b0d16f2c60ea289f2bbbfb88f45"} Jul 23 12:10:55.353701 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:55.353578 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:10:55.354640 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:55.354605 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tcgfq" event={"ID":"88f19386-8325-4469-9859-cd876612dcd6","Type":"ContainerStarted","Data":"39ecbab88609d2550d937bc4e0282b14d0a1de39ffd7ce11943f97845e4554bc"} Jul 23 12:10:55.370545 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:55.370502 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tjdx4" podStartSLOduration=66.370489092 podStartE2EDuration="1m6.370489092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:09:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:55.370288088 +0000 UTC m=+66.962822782" watchObservedRunningTime="2026-07-23 12:10:55.370489092 +0000 UTC m=+66.963023784" Jul 23 12:10:57.147512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:57.147476 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:57.147945 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:57.147526 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:10:57.147945 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:57.147558 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09f3adc9-0ded-4260-bedb-cf97199c0a39-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:10:57.147945 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:57.147584 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls\") pod \"dns-default-2wl6p\" (UID: \"f8597295-ae1a-42e6-ac3f-293915f8a69c\") " pod="openshift-dns/dns-default-2wl6p" Jul 23 12:10:57.147945 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:57.147615 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 12:10:57.147945 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:57.147693 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-rbac-proxy-cm podName:09f3adc9-0ded-4260-bedb-cf97199c0a39 nodeName:}" failed. No retries permitted until 2026-07-23 12:11:05.147672883 +0000 UTC m=+76.740207558 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7z6nv" (UID: "09f3adc9-0ded-4260-bedb-cf97199c0a39") : configmap "kube-rbac-proxy" not found Jul 23 12:10:57.147945 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:57.147720 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 12:10:57.147945 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:57.147725 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 12:10:57.147945 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:57.147744 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5999b54cbc-wg5fw: secret "image-registry-tls" not found Jul 23 12:10:57.147945 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:57.147769 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls podName:f8597295-ae1a-42e6-ac3f-293915f8a69c nodeName:}" failed. No retries permitted until 2026-07-23 12:11:29.147753268 +0000 UTC m=+100.740287955 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls") pod "dns-default-2wl6p" (UID: "f8597295-ae1a-42e6-ac3f-293915f8a69c") : secret "dns-default-metrics-tls" not found Jul 23 12:10:57.147945 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:57.147792 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls podName:c3e44bef-488e-4d15-bd18-6cbbeec2b5eb nodeName:}" failed. No retries permitted until 2026-07-23 12:11:29.147777603 +0000 UTC m=+100.740312287 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls") pod "image-registry-5999b54cbc-wg5fw" (UID: "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb") : secret "image-registry-tls" not found Jul 23 12:10:57.147945 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:57.147732 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 12:10:57.147945 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:10:57.147830 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09f3adc9-0ded-4260-bedb-cf97199c0a39-insights-runtime-extractor-tls podName:09f3adc9-0ded-4260-bedb-cf97199c0a39 nodeName:}" failed. No retries permitted until 2026-07-23 12:11:05.147819866 +0000 UTC m=+76.740354537 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/09f3adc9-0ded-4260-bedb-cf97199c0a39-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7z6nv" (UID: "09f3adc9-0ded-4260-bedb-cf97199c0a39") : secret "insights-runtime-extractor-tls" not found Jul 23 12:10:59.370958 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:59.370922 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tcgfq" event={"ID":"88f19386-8325-4469-9859-cd876612dcd6","Type":"ContainerStarted","Data":"144d3022377921b6f8ed122776fd4d59d2aaf96642b94450ab27be6e5bd12edb"} Jul 23 12:10:59.391019 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:10:59.390975 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tcgfq" podStartSLOduration=66.315589602 podStartE2EDuration="1m10.390961459s" totalImagesPullingTime="4.075371857s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:09:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:10:59.390598361 +0000 UTC m=+70.983133056" watchObservedRunningTime="2026-07-23 12:10:59.390961459 +0000 UTC m=+70.983496151" Jul 23 12:11:05.222999 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:05.222962 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:11:05.223388 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:05.223016 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09f3adc9-0ded-4260-bedb-cf97199c0a39-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:11:05.223528 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:05.223493 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/09f3adc9-0ded-4260-bedb-cf97199c0a39-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:11:05.225252 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:05.225233 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/09f3adc9-0ded-4260-bedb-cf97199c0a39-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7z6nv\" (UID: \"09f3adc9-0ded-4260-bedb-cf97199c0a39\") " pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:11:05.287320 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:05.287286 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7z6nv" Jul 23 12:11:05.433082 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:05.433050 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7z6nv"] Jul 23 12:11:05.436046 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:11:05.436017 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09f3adc9_0ded_4260_bedb_cf97199c0a39.slice/crio-fe4768a3a87efd0dbb21bbc15471205368738ac729ebcd1ccb031be63a474d40 WatchSource:0}: Error finding container fe4768a3a87efd0dbb21bbc15471205368738ac729ebcd1ccb031be63a474d40: Status 404 returned error can't find the container with id fe4768a3a87efd0dbb21bbc15471205368738ac729ebcd1ccb031be63a474d40 Jul 23 12:11:06.395812 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:06.395766 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7z6nv" event={"ID":"09f3adc9-0ded-4260-bedb-cf97199c0a39","Type":"ContainerStarted","Data":"fe4768a3a87efd0dbb21bbc15471205368738ac729ebcd1ccb031be63a474d40"} Jul 23 12:11:10.410619 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:10.410581 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7z6nv" event={"ID":"09f3adc9-0ded-4260-bedb-cf97199c0a39","Type":"ContainerStarted","Data":"7537c7e81e64dd1c6b9f6d2b31c13f22b84c1517c0aabb6ee821bccde7c8ecd4"} Jul 23 12:11:13.396182 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:13.396062 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-5zqgt"] Jul 23 12:11:13.408799 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:13.408765 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-5zqgt" Jul 23 12:11:13.410524 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:13.410496 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-5zqgt"] Jul 23 12:11:13.411917 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:13.411880 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-fb5rb\"" Jul 23 12:11:13.412041 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:13.411945 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 23 12:11:13.412041 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:13.411973 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 23 12:11:13.494100 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:13.494070 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf6d9\" (UniqueName: \"kubernetes.io/projected/e02a2643-257b-4aef-a900-3528d209b039-kube-api-access-kf6d9\") pod \"downloads-5bc67dcd46-5zqgt\" (UID: \"e02a2643-257b-4aef-a900-3528d209b039\") " pod="openshift-console/downloads-5bc67dcd46-5zqgt" Jul 23 12:11:13.594485 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:13.594445 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf6d9\" (UniqueName: \"kubernetes.io/projected/e02a2643-257b-4aef-a900-3528d209b039-kube-api-access-kf6d9\") pod \"downloads-5bc67dcd46-5zqgt\" (UID: \"e02a2643-257b-4aef-a900-3528d209b039\") " pod="openshift-console/downloads-5bc67dcd46-5zqgt" Jul 23 12:11:13.604550 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:13.604525 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf6d9\" (UniqueName: \"kubernetes.io/projected/e02a2643-257b-4aef-a900-3528d209b039-kube-api-access-kf6d9\") pod \"downloads-5bc67dcd46-5zqgt\" (UID: \"e02a2643-257b-4aef-a900-3528d209b039\") " pod="openshift-console/downloads-5bc67dcd46-5zqgt" Jul 23 12:11:13.719558 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:13.719533 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-5zqgt" Jul 23 12:11:13.845071 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:13.845043 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-5zqgt"] Jul 23 12:11:13.848106 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:11:13.848079 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode02a2643_257b_4aef_a900_3528d209b039.slice/crio-391c4d900ab5a13f40d5e7083f854e442770a84b5d0218f411ff9a2c974ce5dc WatchSource:0}: Error finding container 391c4d900ab5a13f40d5e7083f854e442770a84b5d0218f411ff9a2c974ce5dc: Status 404 returned error can't find the container with id 391c4d900ab5a13f40d5e7083f854e442770a84b5d0218f411ff9a2c974ce5dc Jul 23 12:11:14.428155 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:14.428115 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7z6nv" event={"ID":"09f3adc9-0ded-4260-bedb-cf97199c0a39","Type":"ContainerStarted","Data":"4636e06d782d182cf1220f00485c8f6addcdf9c299e04aed6225e58cbfd9232c"} Jul 23 12:11:14.429394 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:14.429363 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-5zqgt" event={"ID":"e02a2643-257b-4aef-a900-3528d209b039","Type":"ContainerStarted","Data":"391c4d900ab5a13f40d5e7083f854e442770a84b5d0218f411ff9a2c974ce5dc"} Jul 23 12:11:18.447390 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:18.447349 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7z6nv" event={"ID":"09f3adc9-0ded-4260-bedb-cf97199c0a39","Type":"ContainerStarted","Data":"68561ed6b1fdab8a81fe26493ab7a2d9291be49fa31892df7d392ea26593633a"} Jul 23 12:11:18.467659 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:18.467609 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7z6nv" podStartSLOduration=17.492443183 podStartE2EDuration="29.467595775s" totalImagesPullingTime="11.975152592s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:10:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:18.466487525 +0000 UTC m=+90.059022233" watchObservedRunningTime="2026-07-23 12:11:18.467595775 +0000 UTC m=+90.060130467" Jul 23 12:11:20.574106 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:20.574075 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gtzbj"] Jul 23 12:11:20.589085 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:20.589015 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gtzbj"] Jul 23 12:11:20.589269 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:20.589125 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gtzbj" Jul 23 12:11:20.591995 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:20.591775 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 23 12:11:20.591995 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:20.591823 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-pjr62\"" Jul 23 12:11:20.755293 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:20.755253 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9a3f74e9-16f8-40f8-8274-01b7b3031ec6-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-gtzbj\" (UID: \"9a3f74e9-16f8-40f8-8274-01b7b3031ec6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gtzbj" Jul 23 12:11:20.856071 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:20.856039 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9a3f74e9-16f8-40f8-8274-01b7b3031ec6-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-gtzbj\" (UID: \"9a3f74e9-16f8-40f8-8274-01b7b3031ec6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gtzbj" Jul 23 12:11:20.856267 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:11:20.856213 2578 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 23 12:11:20.856332 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:11:20.856288 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a3f74e9-16f8-40f8-8274-01b7b3031ec6-tls-certificates podName:9a3f74e9-16f8-40f8-8274-01b7b3031ec6 nodeName:}" failed. No retries permitted until 2026-07-23 12:11:21.356270415 +0000 UTC m=+92.948805090 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/9a3f74e9-16f8-40f8-8274-01b7b3031ec6-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-gtzbj" (UID: "9a3f74e9-16f8-40f8-8274-01b7b3031ec6") : secret "prometheus-operator-admission-webhook-tls" not found Jul 23 12:11:21.360498 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:21.360462 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9a3f74e9-16f8-40f8-8274-01b7b3031ec6-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-gtzbj\" (UID: \"9a3f74e9-16f8-40f8-8274-01b7b3031ec6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gtzbj" Jul 23 12:11:21.363148 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:21.363122 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9a3f74e9-16f8-40f8-8274-01b7b3031ec6-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-gtzbj\" (UID: \"9a3f74e9-16f8-40f8-8274-01b7b3031ec6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gtzbj" Jul 23 12:11:21.503161 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:21.503127 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gtzbj" Jul 23 12:11:21.660962 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:21.660890 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gtzbj"] Jul 23 12:11:21.664380 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:11:21.664351 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a3f74e9_16f8_40f8_8274_01b7b3031ec6.slice/crio-6752d69f370531e50d482a2e3d1fabde24d5d69dec63aecde63f4c17c2a33b15 WatchSource:0}: Error finding container 6752d69f370531e50d482a2e3d1fabde24d5d69dec63aecde63f4c17c2a33b15: Status 404 returned error can't find the container with id 6752d69f370531e50d482a2e3d1fabde24d5d69dec63aecde63f4c17c2a33b15 Jul 23 12:11:22.462202 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.462152 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gtzbj" event={"ID":"9a3f74e9-16f8-40f8-8274-01b7b3031ec6","Type":"ContainerStarted","Data":"6752d69f370531e50d482a2e3d1fabde24d5d69dec63aecde63f4c17c2a33b15"} Jul 23 12:11:22.486273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.486244 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5fcff596c8-xlzvz"] Jul 23 12:11:22.509084 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.509043 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fcff596c8-xlzvz"] Jul 23 12:11:22.509252 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.509159 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:22.512078 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.511944 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 23 12:11:22.512078 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.511969 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 23 12:11:22.512290 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.512154 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-6gcs6\"" Jul 23 12:11:22.513150 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.512991 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 23 12:11:22.513150 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.513002 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 23 12:11:22.513150 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.513076 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 23 12:11:22.673049 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.673013 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-console-oauth-config\") pod \"console-5fcff596c8-xlzvz\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:22.673502 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.673058 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-service-ca\") pod \"console-5fcff596c8-xlzvz\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:22.673502 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.673089 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-oauth-serving-cert\") pod \"console-5fcff596c8-xlzvz\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:22.673502 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.673126 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fkdb\" (UniqueName: \"kubernetes.io/projected/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-kube-api-access-7fkdb\") pod \"console-5fcff596c8-xlzvz\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:22.673502 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.673232 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-console-serving-cert\") pod \"console-5fcff596c8-xlzvz\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:22.673502 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.673270 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-console-config\") pod \"console-5fcff596c8-xlzvz\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:22.774183 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.774103 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-console-serving-cert\") pod \"console-5fcff596c8-xlzvz\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:22.774352 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.774215 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-console-config\") pod \"console-5fcff596c8-xlzvz\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:22.774416 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.774373 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-console-oauth-config\") pod \"console-5fcff596c8-xlzvz\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:22.774416 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.774401 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-service-ca\") pod \"console-5fcff596c8-xlzvz\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:22.774584 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.774434 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-oauth-serving-cert\") pod \"console-5fcff596c8-xlzvz\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:22.775429 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.775011 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-console-config\") pod \"console-5fcff596c8-xlzvz\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:22.775429 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.775069 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fkdb\" (UniqueName: \"kubernetes.io/projected/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-kube-api-access-7fkdb\") pod \"console-5fcff596c8-xlzvz\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:22.775429 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.775099 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-oauth-serving-cert\") pod \"console-5fcff596c8-xlzvz\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:22.775429 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.775286 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-service-ca\") pod \"console-5fcff596c8-xlzvz\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:22.777660 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.777425 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-console-oauth-config\") pod \"console-5fcff596c8-xlzvz\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:22.777785 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.777738 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-console-serving-cert\") pod \"console-5fcff596c8-xlzvz\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:22.790647 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.790616 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fkdb\" (UniqueName: \"kubernetes.io/projected/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-kube-api-access-7fkdb\") pod \"console-5fcff596c8-xlzvz\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:22.822348 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:22.822320 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:23.150335 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:23.150304 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fcff596c8-xlzvz"] Jul 23 12:11:23.154068 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:11:23.154033 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8b43ea3_4eb8_4440_9e8b_0c08522b54b0.slice/crio-be974e9045b5b92746091fa9a3b8bff82d719ef4183254f890b19deff25e7b78 WatchSource:0}: Error finding container be974e9045b5b92746091fa9a3b8bff82d719ef4183254f890b19deff25e7b78: Status 404 returned error can't find the container with id be974e9045b5b92746091fa9a3b8bff82d719ef4183254f890b19deff25e7b78 Jul 23 12:11:23.467060 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:23.467023 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fcff596c8-xlzvz" event={"ID":"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0","Type":"ContainerStarted","Data":"be974e9045b5b92746091fa9a3b8bff82d719ef4183254f890b19deff25e7b78"} Jul 23 12:11:23.468768 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:23.468722 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gtzbj" event={"ID":"9a3f74e9-16f8-40f8-8274-01b7b3031ec6","Type":"ContainerStarted","Data":"9a5d0eeec9b8f1bc4509b67090a345379114e14753a49eab05cafaead58f072a"} Jul 23 12:11:24.472400 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:24.472367 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gtzbj" Jul 23 12:11:24.477578 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:24.477556 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gtzbj" Jul 23 12:11:24.491598 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:24.491553 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-gtzbj" podStartSLOduration=2.80911418 podStartE2EDuration="4.491538851s" totalImagesPullingTime="1.682424671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:11:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:24.490296576 +0000 UTC m=+96.082831270" watchObservedRunningTime="2026-07-23 12:11:24.491538851 +0000 UTC m=+96.084073543" Jul 23 12:11:26.361706 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:26.361665 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tjdx4" Jul 23 12:11:29.019432 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.019397 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9"] Jul 23 12:11:29.044837 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.044720 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9" Jul 23 12:11:29.045504 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.045335 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9"] Jul 23 12:11:29.047223 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.047180 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-vmmhc"] Jul 23 12:11:29.047714 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.047619 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 23 12:11:29.048742 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.048307 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 23 12:11:29.048742 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.048685 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 23 12:11:29.049444 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.049251 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 23 12:11:29.049444 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.049319 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 23 12:11:29.049608 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.049501 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-8ld9m\"" Jul 23 12:11:29.071683 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.071655 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-498sv"] Jul 23 12:11:29.071838 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.071779 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" Jul 23 12:11:29.074846 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.074555 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 23 12:11:29.074846 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.074643 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 23 12:11:29.075390 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.075208 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 23 12:11:29.075390 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.075308 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-qkbnj\"" Jul 23 12:11:29.087785 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.087753 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-vmmhc"] Jul 23 12:11:29.087903 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.087888 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.090922 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.090775 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 23 12:11:29.090922 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.090838 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 23 12:11:29.090922 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.090871 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sc6d5\"" Jul 23 12:11:29.091316 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.091296 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 23 12:11:29.128235 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.128172 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b69a1fe-10ac-4f21-adfc-aad899e19811-metrics-client-ca\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.128499 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.128472 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/72799695-c5dc-498b-863e-07240479551c-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-vmmhc\" (UID: \"72799695-c5dc-498b-863e-07240479551c\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" Jul 23 12:11:29.128634 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.128519 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/72799695-c5dc-498b-863e-07240479551c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-vmmhc\" (UID: \"72799695-c5dc-498b-863e-07240479551c\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" Jul 23 12:11:29.128634 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.128549 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6f94b8f-cf41-495b-812e-4606666aa268-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-cbzx9\" (UID: \"e6f94b8f-cf41-495b-812e-4606666aa268\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9" Jul 23 12:11:29.128738 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.128662 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0b69a1fe-10ac-4f21-adfc-aad899e19811-node-exporter-tls\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.128816 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.128802 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/72799695-c5dc-498b-863e-07240479551c-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-vmmhc\" (UID: \"72799695-c5dc-498b-863e-07240479551c\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" Jul 23 12:11:29.128873 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.128837 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6f94b8f-cf41-495b-812e-4606666aa268-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-cbzx9\" (UID: \"e6f94b8f-cf41-495b-812e-4606666aa268\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9" Jul 23 12:11:29.128873 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.128862 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htpck\" (UniqueName: \"kubernetes.io/projected/72799695-c5dc-498b-863e-07240479551c-kube-api-access-htpck\") pod \"kube-state-metrics-56c5594674-vmmhc\" (UID: \"72799695-c5dc-498b-863e-07240479551c\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" Jul 23 12:11:29.128977 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.128906 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-522xt\" (UniqueName: \"kubernetes.io/projected/0b69a1fe-10ac-4f21-adfc-aad899e19811-kube-api-access-522xt\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.128977 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.128938 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/72799695-c5dc-498b-863e-07240479551c-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-vmmhc\" (UID: \"72799695-c5dc-498b-863e-07240479551c\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" Jul 23 12:11:29.128977 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.128968 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/72799695-c5dc-498b-863e-07240479551c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-vmmhc\" (UID: \"72799695-c5dc-498b-863e-07240479551c\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" Jul 23 12:11:29.129174 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.129000 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0b69a1fe-10ac-4f21-adfc-aad899e19811-node-exporter-accelerators-collector-config\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.129174 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.129115 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6f94b8f-cf41-495b-812e-4606666aa268-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-cbzx9\" (UID: \"e6f94b8f-cf41-495b-812e-4606666aa268\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9" Jul 23 12:11:29.129174 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.129147 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qldh7\" (UniqueName: \"kubernetes.io/projected/e6f94b8f-cf41-495b-812e-4606666aa268-kube-api-access-qldh7\") pod \"openshift-state-metrics-79f6958cdf-cbzx9\" (UID: \"e6f94b8f-cf41-495b-812e-4606666aa268\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9" Jul 23 12:11:29.129390 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.129175 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b69a1fe-10ac-4f21-adfc-aad899e19811-sys\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.129390 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.129226 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b69a1fe-10ac-4f21-adfc-aad899e19811-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.129390 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.129254 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0b69a1fe-10ac-4f21-adfc-aad899e19811-root\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.129390 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.129277 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0b69a1fe-10ac-4f21-adfc-aad899e19811-node-exporter-wtmp\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.129390 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.129309 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0b69a1fe-10ac-4f21-adfc-aad899e19811-node-exporter-textfile\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.230336 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.230284 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b69a1fe-10ac-4f21-adfc-aad899e19811-sys\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.230336 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.230337 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b69a1fe-10ac-4f21-adfc-aad899e19811-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.230563 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.230397 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0b69a1fe-10ac-4f21-adfc-aad899e19811-root\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.230563 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.230482 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0b69a1fe-10ac-4f21-adfc-aad899e19811-node-exporter-wtmp\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.230563 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.230519 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0b69a1fe-10ac-4f21-adfc-aad899e19811-node-exporter-textfile\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.230563 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.230546 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b69a1fe-10ac-4f21-adfc-aad899e19811-metrics-client-ca\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.230798 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.230574 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/72799695-c5dc-498b-863e-07240479551c-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-vmmhc\" (UID: \"72799695-c5dc-498b-863e-07240479551c\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" Jul 23 12:11:29.230798 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.230609 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/72799695-c5dc-498b-863e-07240479551c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-vmmhc\" (UID: \"72799695-c5dc-498b-863e-07240479551c\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" Jul 23 12:11:29.230798 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.230642 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6f94b8f-cf41-495b-812e-4606666aa268-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-cbzx9\" (UID: \"e6f94b8f-cf41-495b-812e-4606666aa268\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9" Jul 23 12:11:29.230798 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.230667 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0b69a1fe-10ac-4f21-adfc-aad899e19811-node-exporter-tls\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.230798 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.230696 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:11:29.230798 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.230732 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/72799695-c5dc-498b-863e-07240479551c-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-vmmhc\" (UID: \"72799695-c5dc-498b-863e-07240479551c\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" Jul 23 12:11:29.230798 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.230761 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6f94b8f-cf41-495b-812e-4606666aa268-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-cbzx9\" (UID: \"e6f94b8f-cf41-495b-812e-4606666aa268\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9" Jul 23 12:11:29.230798 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.230784 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htpck\" (UniqueName: \"kubernetes.io/projected/72799695-c5dc-498b-863e-07240479551c-kube-api-access-htpck\") pod \"kube-state-metrics-56c5594674-vmmhc\" (UID: \"72799695-c5dc-498b-863e-07240479551c\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" Jul 23 12:11:29.231386 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.230813 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls\") pod \"dns-default-2wl6p\" (UID: \"f8597295-ae1a-42e6-ac3f-293915f8a69c\") " pod="openshift-dns/dns-default-2wl6p" Jul 23 12:11:29.231386 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.230838 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-522xt\" (UniqueName: \"kubernetes.io/projected/0b69a1fe-10ac-4f21-adfc-aad899e19811-kube-api-access-522xt\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.231386 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.230865 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/72799695-c5dc-498b-863e-07240479551c-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-vmmhc\" (UID: \"72799695-c5dc-498b-863e-07240479551c\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" Jul 23 12:11:29.231386 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.230894 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/72799695-c5dc-498b-863e-07240479551c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-vmmhc\" (UID: \"72799695-c5dc-498b-863e-07240479551c\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" Jul 23 12:11:29.231386 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.230928 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0b69a1fe-10ac-4f21-adfc-aad899e19811-node-exporter-accelerators-collector-config\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.231386 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.230980 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6f94b8f-cf41-495b-812e-4606666aa268-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-cbzx9\" (UID: \"e6f94b8f-cf41-495b-812e-4606666aa268\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9" Jul 23 12:11:29.231386 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.231006 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b69a1fe-10ac-4f21-adfc-aad899e19811-sys\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.231386 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.231141 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0b69a1fe-10ac-4f21-adfc-aad899e19811-node-exporter-wtmp\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.231386 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.231008 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qldh7\" (UniqueName: \"kubernetes.io/projected/e6f94b8f-cf41-495b-812e-4606666aa268-kube-api-access-qldh7\") pod \"openshift-state-metrics-79f6958cdf-cbzx9\" (UID: \"e6f94b8f-cf41-495b-812e-4606666aa268\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9" Jul 23 12:11:29.231747 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.231413 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0b69a1fe-10ac-4f21-adfc-aad899e19811-root\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.231747 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:11:29.231510 2578 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 23 12:11:29.231747 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:11:29.231579 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6f94b8f-cf41-495b-812e-4606666aa268-openshift-state-metrics-tls podName:e6f94b8f-cf41-495b-812e-4606666aa268 nodeName:}" failed. No retries permitted until 2026-07-23 12:11:29.731559996 +0000 UTC m=+101.324094672 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/e6f94b8f-cf41-495b-812e-4606666aa268-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-cbzx9" (UID: "e6f94b8f-cf41-495b-812e-4606666aa268") : secret "openshift-state-metrics-tls" not found Jul 23 12:11:29.233980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.232534 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/72799695-c5dc-498b-863e-07240479551c-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-vmmhc\" (UID: \"72799695-c5dc-498b-863e-07240479551c\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" Jul 23 12:11:29.233980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.232615 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b69a1fe-10ac-4f21-adfc-aad899e19811-metrics-client-ca\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.233980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.231509 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0b69a1fe-10ac-4f21-adfc-aad899e19811-node-exporter-textfile\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.233980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.233086 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0b69a1fe-10ac-4f21-adfc-aad899e19811-node-exporter-accelerators-collector-config\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.233980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.233116 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6f94b8f-cf41-495b-812e-4606666aa268-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-cbzx9\" (UID: \"e6f94b8f-cf41-495b-812e-4606666aa268\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9" Jul 23 12:11:29.233980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.233375 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b69a1fe-10ac-4f21-adfc-aad899e19811-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.233980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.233919 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/72799695-c5dc-498b-863e-07240479551c-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-vmmhc\" (UID: \"72799695-c5dc-498b-863e-07240479551c\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" Jul 23 12:11:29.234447 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.234045 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/72799695-c5dc-498b-863e-07240479551c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-vmmhc\" (UID: \"72799695-c5dc-498b-863e-07240479551c\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" Jul 23 12:11:29.235033 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.234926 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/72799695-c5dc-498b-863e-07240479551c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-vmmhc\" (UID: \"72799695-c5dc-498b-863e-07240479551c\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" Jul 23 12:11:29.235257 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.235231 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/72799695-c5dc-498b-863e-07240479551c-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-vmmhc\" (UID: \"72799695-c5dc-498b-863e-07240479551c\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" Jul 23 12:11:29.237020 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.236997 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls\") pod \"image-registry-5999b54cbc-wg5fw\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:11:29.237576 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.237557 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f8597295-ae1a-42e6-ac3f-293915f8a69c-metrics-tls\") pod \"dns-default-2wl6p\" (UID: \"f8597295-ae1a-42e6-ac3f-293915f8a69c\") " pod="openshift-dns/dns-default-2wl6p" Jul 23 12:11:29.239285 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.239266 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0b69a1fe-10ac-4f21-adfc-aad899e19811-node-exporter-tls\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.239791 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.239766 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6f94b8f-cf41-495b-812e-4606666aa268-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-cbzx9\" (UID: \"e6f94b8f-cf41-495b-812e-4606666aa268\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9" Jul 23 12:11:29.241328 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.241287 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qldh7\" (UniqueName: \"kubernetes.io/projected/e6f94b8f-cf41-495b-812e-4606666aa268-kube-api-access-qldh7\") pod \"openshift-state-metrics-79f6958cdf-cbzx9\" (UID: \"e6f94b8f-cf41-495b-812e-4606666aa268\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9" Jul 23 12:11:29.243708 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.243647 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htpck\" (UniqueName: \"kubernetes.io/projected/72799695-c5dc-498b-863e-07240479551c-kube-api-access-htpck\") pod \"kube-state-metrics-56c5594674-vmmhc\" (UID: \"72799695-c5dc-498b-863e-07240479551c\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" Jul 23 12:11:29.243815 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.243737 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-522xt\" (UniqueName: \"kubernetes.io/projected/0b69a1fe-10ac-4f21-adfc-aad899e19811-kube-api-access-522xt\") pod \"node-exporter-498sv\" (UID: \"0b69a1fe-10ac-4f21-adfc-aad899e19811\") " pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.384293 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.384256 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" Jul 23 12:11:29.399416 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.399389 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-498sv" Jul 23 12:11:29.504176 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.504146 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xdbs4\"" Jul 23 12:11:29.512509 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.512488 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:11:29.526165 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.526141 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dbmrb\"" Jul 23 12:11:29.533517 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.533492 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2wl6p" Jul 23 12:11:29.735545 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.735510 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6f94b8f-cf41-495b-812e-4606666aa268-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-cbzx9\" (UID: \"e6f94b8f-cf41-495b-812e-4606666aa268\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9" Jul 23 12:11:29.738165 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.738138 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6f94b8f-cf41-495b-812e-4606666aa268-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-cbzx9\" (UID: \"e6f94b8f-cf41-495b-812e-4606666aa268\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9" Jul 23 12:11:29.957396 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:29.957362 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9" Jul 23 12:11:30.778696 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:11:30.778662 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b69a1fe_10ac_4f21_adfc_aad899e19811.slice/crio-68cfb5aa5138f41cb749496ab70f9a75e9cf0e6533acc3e020a3d7612eedefa9 WatchSource:0}: Error finding container 68cfb5aa5138f41cb749496ab70f9a75e9cf0e6533acc3e020a3d7612eedefa9: Status 404 returned error can't find the container with id 68cfb5aa5138f41cb749496ab70f9a75e9cf0e6533acc3e020a3d7612eedefa9 Jul 23 12:11:30.923756 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:30.923723 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2wl6p"] Jul 23 12:11:30.942343 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:30.942316 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-vmmhc"] Jul 23 12:11:30.953436 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:11:30.953407 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8597295_ae1a_42e6_ac3f_293915f8a69c.slice/crio-a23e8e11b0ce6bf790162d1d3805dec3a408c5599fc32bd5cb1667f4ac961f6c WatchSource:0}: Error finding container a23e8e11b0ce6bf790162d1d3805dec3a408c5599fc32bd5cb1667f4ac961f6c: Status 404 returned error can't find the container with id a23e8e11b0ce6bf790162d1d3805dec3a408c5599fc32bd5cb1667f4ac961f6c Jul 23 12:11:30.953968 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:11:30.953946 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72799695_c5dc_498b_863e_07240479551c.slice/crio-dede03f302c666b76618ebb129dcd3e74bf2fed33d355883f67f8d25c7dcc868 WatchSource:0}: Error finding container dede03f302c666b76618ebb129dcd3e74bf2fed33d355883f67f8d25c7dcc868: Status 404 returned error can't find the container with id dede03f302c666b76618ebb129dcd3e74bf2fed33d355883f67f8d25c7dcc868 Jul 23 12:11:30.958830 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:30.958805 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9"] Jul 23 12:11:30.961986 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:11:30.961955 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6f94b8f_cf41_495b_812e_4606666aa268.slice/crio-17c47ee25f32c70c43225839f5adabe2dcfe70fa4d5cd05de0f550343b66ba78 WatchSource:0}: Error finding container 17c47ee25f32c70c43225839f5adabe2dcfe70fa4d5cd05de0f550343b66ba78: Status 404 returned error can't find the container with id 17c47ee25f32c70c43225839f5adabe2dcfe70fa4d5cd05de0f550343b66ba78 Jul 23 12:11:30.987242 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:30.987215 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5999b54cbc-wg5fw"] Jul 23 12:11:30.989905 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:11:30.989884 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3e44bef_488e_4d15_bd18_6cbbeec2b5eb.slice/crio-c461e3a58b48bf918048371eab3f426e5cb30f16f5e46dbfb44178af9afe86a4 WatchSource:0}: Error finding container c461e3a58b48bf918048371eab3f426e5cb30f16f5e46dbfb44178af9afe86a4: Status 404 returned error can't find the container with id c461e3a58b48bf918048371eab3f426e5cb30f16f5e46dbfb44178af9afe86a4 Jul 23 12:11:31.471615 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.471573 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-758588cc9d-ncpbg"] Jul 23 12:11:31.474949 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.474247 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.493313 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.493174 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-758588cc9d-ncpbg"] Jul 23 12:11:31.493951 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.493928 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 23 12:11:31.526012 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.525706 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" event={"ID":"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb","Type":"ContainerStarted","Data":"c256f976326e096274d34708f47381e6bf24c79eac3153de0c9808fc8774016e"} Jul 23 12:11:31.526012 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.525800 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" event={"ID":"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb","Type":"ContainerStarted","Data":"c461e3a58b48bf918048371eab3f426e5cb30f16f5e46dbfb44178af9afe86a4"} Jul 23 12:11:31.526012 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.525959 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:11:31.533171 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.533137 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9" event={"ID":"e6f94b8f-cf41-495b-812e-4606666aa268","Type":"ContainerStarted","Data":"e9121128880c6c2eb08bd7078f0262401c8dd6264f880157b519b543bfc66679"} Jul 23 12:11:31.533481 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.533461 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9" event={"ID":"e6f94b8f-cf41-495b-812e-4606666aa268","Type":"ContainerStarted","Data":"ecafd7cf4e770a0a5e7ea945e68a823380a38fd77b29d1a97958e8120ea59471"} Jul 23 12:11:31.533616 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.533602 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9" event={"ID":"e6f94b8f-cf41-495b-812e-4606666aa268","Type":"ContainerStarted","Data":"17c47ee25f32c70c43225839f5adabe2dcfe70fa4d5cd05de0f550343b66ba78"} Jul 23 12:11:31.537019 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.536994 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-5zqgt" event={"ID":"e02a2643-257b-4aef-a900-3528d209b039","Type":"ContainerStarted","Data":"c36f070242e78752e24f324ccf9bcbbf62ea741a2e501ec507d33427098b883c"} Jul 23 12:11:31.537231 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.537216 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-5zqgt" Jul 23 12:11:31.539813 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.539788 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" event={"ID":"72799695-c5dc-498b-863e-07240479551c","Type":"ContainerStarted","Data":"dede03f302c666b76618ebb129dcd3e74bf2fed33d355883f67f8d25c7dcc868"} Jul 23 12:11:31.541598 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.541570 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2wl6p" event={"ID":"f8597295-ae1a-42e6-ac3f-293915f8a69c","Type":"ContainerStarted","Data":"a23e8e11b0ce6bf790162d1d3805dec3a408c5599fc32bd5cb1667f4ac961f6c"} Jul 23 12:11:31.543886 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.543820 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-498sv" event={"ID":"0b69a1fe-10ac-4f21-adfc-aad899e19811","Type":"ContainerStarted","Data":"68cfb5aa5138f41cb749496ab70f9a75e9cf0e6533acc3e020a3d7612eedefa9"} Jul 23 12:11:31.548384 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.548329 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-5zqgt" Jul 23 12:11:31.552885 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.549734 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-trusted-ca-bundle\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.552885 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.549772 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d147d50d-300b-4f7c-986f-06880311e014-console-serving-cert\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.552885 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.549841 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-service-ca\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.552885 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.549871 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-oauth-serving-cert\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.552885 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.549905 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d147d50d-300b-4f7c-986f-06880311e014-console-oauth-config\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.552885 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.550044 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjhkt\" (UniqueName: \"kubernetes.io/projected/d147d50d-300b-4f7c-986f-06880311e014-kube-api-access-gjhkt\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.552885 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.550128 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-console-config\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.555660 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.555613 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" podStartSLOduration=102.555595898 podStartE2EDuration="1m42.555595898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:09:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:31.553857554 +0000 UTC m=+103.146392248" watchObservedRunningTime="2026-07-23 12:11:31.555595898 +0000 UTC m=+103.148130592" Jul 23 12:11:31.651074 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.651025 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjhkt\" (UniqueName: \"kubernetes.io/projected/d147d50d-300b-4f7c-986f-06880311e014-kube-api-access-gjhkt\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.651252 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.651099 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-console-config\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.651252 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.651159 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-trusted-ca-bundle\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.651252 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.651186 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d147d50d-300b-4f7c-986f-06880311e014-console-serving-cert\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.651252 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.651248 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-service-ca\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.651474 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.651271 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-oauth-serving-cert\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.651474 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.651304 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d147d50d-300b-4f7c-986f-06880311e014-console-oauth-config\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.652756 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.652612 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-trusted-ca-bundle\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.652756 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.652656 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-service-ca\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.653932 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.653870 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-oauth-serving-cert\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.653932 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.653919 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-console-config\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.656325 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.656285 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d147d50d-300b-4f7c-986f-06880311e014-console-oauth-config\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.657128 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.657050 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d147d50d-300b-4f7c-986f-06880311e014-console-serving-cert\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.665057 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.664973 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjhkt\" (UniqueName: \"kubernetes.io/projected/d147d50d-300b-4f7c-986f-06880311e014-kube-api-access-gjhkt\") pod \"console-758588cc9d-ncpbg\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:31.791951 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:31.791758 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:32.654532 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:32.653849 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-5zqgt" podStartSLOduration=2.494031356 podStartE2EDuration="19.653825555s" totalImagesPullingTime="17.159794199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:11:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:31.573946788 +0000 UTC m=+103.166481483" watchObservedRunningTime="2026-07-23 12:11:32.653825555 +0000 UTC m=+104.246360229" Jul 23 12:11:32.656408 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:32.656331 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-758588cc9d-ncpbg"] Jul 23 12:11:33.561893 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.561817 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-758588cc9d-ncpbg" event={"ID":"d147d50d-300b-4f7c-986f-06880311e014","Type":"ContainerStarted","Data":"1c0df5bf5d63bda8ec8a8d74cdff8cc42f25155b2bc1d95f27c5446ae39f18d6"} Jul 23 12:11:33.566797 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.565390 2578 generic.go:356] "Generic (PLEG): container finished" podID="0b69a1fe-10ac-4f21-adfc-aad899e19811" containerID="779afdfa9dfcaf46632d29b15903d458ad1bc479dc49b0bfbe32d120e1ade687" exitCode=0 Jul 23 12:11:33.566797 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.566522 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-498sv" event={"ID":"0b69a1fe-10ac-4f21-adfc-aad899e19811","Type":"ContainerDied","Data":"779afdfa9dfcaf46632d29b15903d458ad1bc479dc49b0bfbe32d120e1ade687"} Jul 23 12:11:33.608648 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.608611 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-749c994bb4-ppwqt"] Jul 23 12:11:33.633465 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.631676 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-749c994bb4-ppwqt"] Jul 23 12:11:33.633465 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.631785 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.640572 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.636316 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 23 12:11:33.640572 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.636828 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 23 12:11:33.640572 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.637092 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 23 12:11:33.640572 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.637320 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-jj986\"" Jul 23 12:11:33.640572 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.638598 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 23 12:11:33.640572 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.638880 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-earfq0ii5sp2a\"" Jul 23 12:11:33.677503 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.677463 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-audit-log\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.677690 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.677536 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-client-ca-bundle\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.677690 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.677593 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqhsf\" (UniqueName: \"kubernetes.io/projected/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-kube-api-access-dqhsf\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.677690 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.677620 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.677845 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.677721 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-secret-metrics-server-tls\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.677845 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.677775 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-secret-metrics-server-client-certs\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.677967 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.677846 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-metrics-server-audit-profiles\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.778733 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.778648 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-secret-metrics-server-tls\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.778733 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.778707 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-secret-metrics-server-client-certs\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.778938 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.778762 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-metrics-server-audit-profiles\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.778938 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.778795 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-audit-log\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.778938 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.778836 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-client-ca-bundle\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.778938 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.778884 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqhsf\" (UniqueName: \"kubernetes.io/projected/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-kube-api-access-dqhsf\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.778938 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.778912 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.780068 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.779744 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-audit-log\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.781124 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.781088 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-metrics-server-audit-profiles\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.781647 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.781620 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.785345 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.785304 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-secret-metrics-server-client-certs\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.786286 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.786262 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-client-ca-bundle\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.786603 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.786547 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-secret-metrics-server-tls\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.796548 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.796495 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqhsf\" (UniqueName: \"kubernetes.io/projected/a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25-kube-api-access-dqhsf\") pod \"metrics-server-749c994bb4-ppwqt\" (UID: \"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25\") " pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.846441 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.845897 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnx47"] Jul 23 12:11:33.863408 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.863374 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnx47"] Jul 23 12:11:33.863568 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.863514 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnx47" Jul 23 12:11:33.867622 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.867276 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-lhlth\"" Jul 23 12:11:33.867622 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.867537 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 23 12:11:33.950793 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.950538 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:33.980931 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.980710 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlpjz\" (UniqueName: \"kubernetes.io/projected/e84909e6-0111-46e7-9ca1-78b5f2cef807-kube-api-access-dlpjz\") pod \"monitoring-plugin-7b7b7bc8bf-wnx47\" (UID: \"e84909e6-0111-46e7-9ca1-78b5f2cef807\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnx47" Jul 23 12:11:33.980931 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:33.980808 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e84909e6-0111-46e7-9ca1-78b5f2cef807-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-wnx47\" (UID: \"e84909e6-0111-46e7-9ca1-78b5f2cef807\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnx47" Jul 23 12:11:34.081848 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.081768 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e84909e6-0111-46e7-9ca1-78b5f2cef807-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-wnx47\" (UID: \"e84909e6-0111-46e7-9ca1-78b5f2cef807\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnx47" Jul 23 12:11:34.082008 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.081881 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlpjz\" (UniqueName: \"kubernetes.io/projected/e84909e6-0111-46e7-9ca1-78b5f2cef807-kube-api-access-dlpjz\") pod \"monitoring-plugin-7b7b7bc8bf-wnx47\" (UID: \"e84909e6-0111-46e7-9ca1-78b5f2cef807\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnx47" Jul 23 12:11:34.082417 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:11:34.082326 2578 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 23 12:11:34.082417 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:11:34.082398 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e84909e6-0111-46e7-9ca1-78b5f2cef807-monitoring-plugin-cert podName:e84909e6-0111-46e7-9ca1-78b5f2cef807 nodeName:}" failed. No retries permitted until 2026-07-23 12:11:34.58237841 +0000 UTC m=+106.174913083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/e84909e6-0111-46e7-9ca1-78b5f2cef807-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-wnx47" (UID: "e84909e6-0111-46e7-9ca1-78b5f2cef807") : secret "monitoring-plugin-cert" not found Jul 23 12:11:34.098596 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.098292 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlpjz\" (UniqueName: \"kubernetes.io/projected/e84909e6-0111-46e7-9ca1-78b5f2cef807-kube-api-access-dlpjz\") pod \"monitoring-plugin-7b7b7bc8bf-wnx47\" (UID: \"e84909e6-0111-46e7-9ca1-78b5f2cef807\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnx47" Jul 23 12:11:34.288028 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.287730 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-644dc889b8-8d7v4"] Jul 23 12:11:34.319530 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.319426 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5999b54cbc-wg5fw"] Jul 23 12:11:34.319530 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.319480 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-644dc889b8-8d7v4"] Jul 23 12:11:34.319771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.319586 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.322961 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.322937 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-tmgmd\"" Jul 23 12:11:34.323236 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.323218 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 23 12:11:34.323488 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.323469 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 23 12:11:34.326124 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.324877 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 23 12:11:34.326124 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.325087 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 23 12:11:34.326124 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.325498 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 23 12:11:34.333788 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.333734 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 23 12:11:34.383897 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.383864 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/826ef539-e43c-48ac-ae5a-9545d8567801-secret-telemeter-client\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.384186 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.384164 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwv92\" (UniqueName: \"kubernetes.io/projected/826ef539-e43c-48ac-ae5a-9545d8567801-kube-api-access-nwv92\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.384435 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.384408 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/826ef539-e43c-48ac-ae5a-9545d8567801-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.384536 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.384447 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/826ef539-e43c-48ac-ae5a-9545d8567801-metrics-client-ca\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.384536 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.384471 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/826ef539-e43c-48ac-ae5a-9545d8567801-telemeter-trusted-ca-bundle\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.384645 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.384507 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/826ef539-e43c-48ac-ae5a-9545d8567801-serving-certs-ca-bundle\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.384645 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.384573 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/826ef539-e43c-48ac-ae5a-9545d8567801-telemeter-client-tls\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.384645 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.384608 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/826ef539-e43c-48ac-ae5a-9545d8567801-federate-client-tls\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.485484 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.485324 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/826ef539-e43c-48ac-ae5a-9545d8567801-secret-telemeter-client\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.485484 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.485404 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwv92\" (UniqueName: \"kubernetes.io/projected/826ef539-e43c-48ac-ae5a-9545d8567801-kube-api-access-nwv92\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.485484 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.485480 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/826ef539-e43c-48ac-ae5a-9545d8567801-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.486009 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.485509 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/826ef539-e43c-48ac-ae5a-9545d8567801-metrics-client-ca\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.486009 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.485530 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/826ef539-e43c-48ac-ae5a-9545d8567801-telemeter-trusted-ca-bundle\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.486009 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.485562 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/826ef539-e43c-48ac-ae5a-9545d8567801-serving-certs-ca-bundle\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.486009 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.485584 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/826ef539-e43c-48ac-ae5a-9545d8567801-telemeter-client-tls\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.486009 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.485614 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/826ef539-e43c-48ac-ae5a-9545d8567801-federate-client-tls\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.486634 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.486609 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/826ef539-e43c-48ac-ae5a-9545d8567801-metrics-client-ca\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.489306 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.487500 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/826ef539-e43c-48ac-ae5a-9545d8567801-serving-certs-ca-bundle\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.489306 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.488392 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/826ef539-e43c-48ac-ae5a-9545d8567801-telemeter-trusted-ca-bundle\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.493523 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.493482 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/826ef539-e43c-48ac-ae5a-9545d8567801-telemeter-client-tls\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.494329 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.494287 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/826ef539-e43c-48ac-ae5a-9545d8567801-secret-telemeter-client\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.494481 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.494447 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/826ef539-e43c-48ac-ae5a-9545d8567801-federate-client-tls\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.494925 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.494885 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/826ef539-e43c-48ac-ae5a-9545d8567801-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.513536 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.513511 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwv92\" (UniqueName: \"kubernetes.io/projected/826ef539-e43c-48ac-ae5a-9545d8567801-kube-api-access-nwv92\") pod \"telemeter-client-644dc889b8-8d7v4\" (UID: \"826ef539-e43c-48ac-ae5a-9545d8567801\") " pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.586835 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.586754 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e84909e6-0111-46e7-9ca1-78b5f2cef807-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-wnx47\" (UID: \"e84909e6-0111-46e7-9ca1-78b5f2cef807\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnx47" Jul 23 12:11:34.590045 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.589992 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e84909e6-0111-46e7-9ca1-78b5f2cef807-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-wnx47\" (UID: \"e84909e6-0111-46e7-9ca1-78b5f2cef807\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnx47" Jul 23 12:11:34.645180 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.644946 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" Jul 23 12:11:34.778947 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:34.778911 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnx47" Jul 23 12:11:35.442014 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.439682 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 12:11:35.463153 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.463118 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.471222 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.468356 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 23 12:11:35.471222 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.468599 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 23 12:11:35.471222 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.468833 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 23 12:11:35.471222 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.469022 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 23 12:11:35.471222 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.470700 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 23 12:11:35.471222 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.470906 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-cl5wk\"" Jul 23 12:11:35.471222 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.471154 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 23 12:11:35.476512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.472554 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 23 12:11:35.476512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.473469 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 12:11:35.476512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.473811 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 23 12:11:35.476512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.473986 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 23 12:11:35.476512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.474040 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2peqj2isnobmm\"" Jul 23 12:11:35.476512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.473989 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 23 12:11:35.476512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.474237 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 23 12:11:35.476512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.474440 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 23 12:11:35.476512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.475365 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 23 12:11:35.481385 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.478659 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 23 12:11:35.498016 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.497987 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.498285 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.498264 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rswl\" (UniqueName: \"kubernetes.io/projected/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-kube-api-access-7rswl\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.498502 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.498483 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.498720 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.498658 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.499488 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.498854 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-config\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.499488 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.498889 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.499488 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.498918 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.499488 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.498941 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.499488 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.498976 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.499488 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.498998 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.499488 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.499017 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.499488 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.499053 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.499488 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.499083 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.499488 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.499111 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.499488 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.499148 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.499488 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.499186 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-web-config\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.499488 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.499228 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.500855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.499258 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-config-out\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.500855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.499283 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.500855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.499314 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.600118 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.600064 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-web-config\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.600118 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.600106 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.600980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.600137 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-config-out\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.600980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.600166 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.600980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.600218 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.600980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.600283 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.600980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.600308 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rswl\" (UniqueName: \"kubernetes.io/projected/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-kube-api-access-7rswl\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.600980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.600355 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.600980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.600394 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.600980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.600438 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-config\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.600980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.600463 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.600980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.600492 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.600980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.600515 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.600980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.600551 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.600980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.600577 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.600980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.600905 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.601633 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.600966 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.601633 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.600993 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.601633 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.601028 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.601633 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.601064 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.602527 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.602503 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.602696 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.602504 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.603309 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.602822 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.603309 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.602944 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.603473 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.603407 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.607499 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.606133 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.607499 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.606845 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.607499 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.607256 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-web-config\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.607707 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.607545 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-config-out\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.609377 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.609321 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.610112 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.610089 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.610430 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.610385 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.611470 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.611430 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-config\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.612669 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.612629 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.612669 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.612636 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rswl\" (UniqueName: \"kubernetes.io/projected/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-kube-api-access-7rswl\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.613459 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.613436 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.613459 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.613449 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.613875 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.613853 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.614908 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.614887 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.616153 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.616117 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:35.784730 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:35.784615 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:37.289768 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:37.289727 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnx47"] Jul 23 12:11:37.293639 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:37.293331 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fcff596c8-xlzvz"] Jul 23 12:11:37.295720 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:11:37.295694 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode84909e6_0111_46e7_9ca1_78b5f2cef807.slice/crio-9e84bc79a55f5185a9f0d5649b0d1ee28079e666845c4e68d4e1bc034cf1aecd WatchSource:0}: Error finding container 9e84bc79a55f5185a9f0d5649b0d1ee28079e666845c4e68d4e1bc034cf1aecd: Status 404 returned error can't find the container with id 9e84bc79a55f5185a9f0d5649b0d1ee28079e666845c4e68d4e1bc034cf1aecd Jul 23 12:11:37.544173 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:37.544120 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-749c994bb4-ppwqt"] Jul 23 12:11:37.547332 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:37.547185 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-644dc889b8-8d7v4"] Jul 23 12:11:37.549607 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:37.549556 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 12:11:37.594567 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:37.592721 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" event={"ID":"826ef539-e43c-48ac-ae5a-9545d8567801","Type":"ContainerStarted","Data":"e6fa9a01bea45d12581590f6a591925c9bfb2d46645b0c7dd41c1b7dd8e60642"} Jul 23 12:11:37.601270 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:37.601153 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" event={"ID":"72799695-c5dc-498b-863e-07240479551c","Type":"ContainerStarted","Data":"adc8f314c2d7e9b603d20e5a1dee103ee8ea087313f80f05aa9c5dc78c695f55"} Jul 23 12:11:37.605472 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:37.603515 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" event={"ID":"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25","Type":"ContainerStarted","Data":"746ea94be7cee75178937eef525f40374dcb640d9114370960a8fe763f1523b1"} Jul 23 12:11:37.612033 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:37.610345 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2wl6p" event={"ID":"f8597295-ae1a-42e6-ac3f-293915f8a69c","Type":"ContainerStarted","Data":"a1f2d32c9c39f0c797878ce20b2295dfd008599a87dc00e93456888112afa46d"} Jul 23 12:11:37.612975 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:37.612876 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3eebdce5-e954-45e3-bccd-4cd4a4aa101a","Type":"ContainerStarted","Data":"d8d700c3ce165d0c724507443c138eb9a485bcef62ab5056e4dfc0a61e3a6200"} Jul 23 12:11:37.617578 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:37.617507 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-498sv" event={"ID":"0b69a1fe-10ac-4f21-adfc-aad899e19811","Type":"ContainerStarted","Data":"ee58f0987b174d10082449f1ae412ffaec5927e78065ae9b491821c90694822c"} Jul 23 12:11:37.617578 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:37.617556 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-498sv" event={"ID":"0b69a1fe-10ac-4f21-adfc-aad899e19811","Type":"ContainerStarted","Data":"8949cb8443f418d50f35bc013972b506cec826bbd69e010f000a22ee294c7c09"} Jul 23 12:11:37.619322 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:37.619274 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnx47" event={"ID":"e84909e6-0111-46e7-9ca1-78b5f2cef807","Type":"ContainerStarted","Data":"9e84bc79a55f5185a9f0d5649b0d1ee28079e666845c4e68d4e1bc034cf1aecd"} Jul 23 12:11:37.621749 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:37.621631 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fcff596c8-xlzvz" event={"ID":"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0","Type":"ContainerStarted","Data":"7a94102688536c74868536ed0894a7ee392aa9dd424b2b053e46974e1401d305"} Jul 23 12:11:37.625145 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:37.625093 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9" event={"ID":"e6f94b8f-cf41-495b-812e-4606666aa268","Type":"ContainerStarted","Data":"b1c8da85a41a550f06b7fb06d7cdcfb1f8c99ac8da66eecaec240011e9d8401a"} Jul 23 12:11:37.643877 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:37.643836 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-498sv" podStartSLOduration=6.439071071 podStartE2EDuration="8.64382291s" totalImagesPullingTime="2.098906343s" totalInitContainerRuntime="105.845496ms" isStatefulPod=true podCreationTimestamp="2026-07-23 12:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:37.642476249 +0000 UTC m=+109.235010943" watchObservedRunningTime="2026-07-23 12:11:37.64382291 +0000 UTC m=+109.236357607" Jul 23 12:11:37.667862 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:37.666833 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-cbzx9" podStartSLOduration=3.730654556 podStartE2EDuration="9.666812161s" totalImagesPullingTime="5.936157605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:37.665366564 +0000 UTC m=+109.257901258" watchObservedRunningTime="2026-07-23 12:11:37.666812161 +0000 UTC m=+109.259346855" Jul 23 12:11:37.690762 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:37.690319 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5fcff596c8-xlzvz" podStartSLOduration=1.7776953949999998 podStartE2EDuration="15.690297754s" totalImagesPullingTime="13.912602359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:37.688484686 +0000 UTC m=+109.281019382" watchObservedRunningTime="2026-07-23 12:11:37.690297754 +0000 UTC m=+109.282832449" Jul 23 12:11:38.645849 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:38.644867 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" event={"ID":"72799695-c5dc-498b-863e-07240479551c","Type":"ContainerStarted","Data":"4463abeb910f9eb41ab0278600c469b779b8625611e547ec067db2456dad63d7"} Jul 23 12:11:38.645849 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:38.644973 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" event={"ID":"72799695-c5dc-498b-863e-07240479551c","Type":"ContainerStarted","Data":"62f1e5bfccba6f91cc96a314d64948173a8631a027648bce1da33956f0e8bb2e"} Jul 23 12:11:38.649114 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:38.648830 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-758588cc9d-ncpbg" event={"ID":"d147d50d-300b-4f7c-986f-06880311e014","Type":"ContainerStarted","Data":"5922c80dc4f628c1a388d04ce2be759db1238472ba12036c48ff5da88ff9492c"} Jul 23 12:11:38.652699 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:38.652673 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2wl6p" event={"ID":"f8597295-ae1a-42e6-ac3f-293915f8a69c","Type":"ContainerStarted","Data":"d2cfb01ac05326a6e9cc5a9d61b76af02a0a8e75849a091d883cd7ad559e6905"} Jul 23 12:11:38.653939 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:38.653720 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2wl6p" Jul 23 12:11:38.672594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:38.671782 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-vmmhc" podStartSLOduration=3.5680329779999997 podStartE2EDuration="9.671762545s" totalImagesPullingTime="6.103729567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:38.668226028 +0000 UTC m=+110.260760719" watchObservedRunningTime="2026-07-23 12:11:38.671762545 +0000 UTC m=+110.264297239" Jul 23 12:11:38.691308 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:38.691075 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-758588cc9d-ncpbg" podStartSLOduration=3.080423157 podStartE2EDuration="7.691055242s" totalImagesPullingTime="4.610632085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:11:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:38.688951925 +0000 UTC m=+110.281486619" watchObservedRunningTime="2026-07-23 12:11:38.691055242 +0000 UTC m=+110.283589936" Jul 23 12:11:41.671641 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:41.671600 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" event={"ID":"a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25","Type":"ContainerStarted","Data":"501b0b598e7334eb8ee7ada5e28b7f7ca068b4fcc74eecaea995e625e887584b"} Jul 23 12:11:41.674330 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:41.674301 2578 generic.go:356] "Generic (PLEG): container finished" podID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerID="9167ccd86b3b8d648cc6cb31e81e37b727ef4311e98f7dcaa0d822e67fb6705f" exitCode=0 Jul 23 12:11:41.674475 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:41.674388 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3eebdce5-e954-45e3-bccd-4cd4a4aa101a","Type":"ContainerDied","Data":"9167ccd86b3b8d648cc6cb31e81e37b727ef4311e98f7dcaa0d822e67fb6705f"} Jul 23 12:11:41.676446 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:41.676187 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnx47" event={"ID":"e84909e6-0111-46e7-9ca1-78b5f2cef807","Type":"ContainerStarted","Data":"1857bfb036baf74c7428898a824313fb7285a62c3e060c2642c0f40cda5680b0"} Jul 23 12:11:41.676609 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:41.676590 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnx47" Jul 23 12:11:41.678866 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:41.678844 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" event={"ID":"826ef539-e43c-48ac-ae5a-9545d8567801","Type":"ContainerStarted","Data":"ec2c2c307a60afe80a865e8385556d0bb945a8ca89c4f714d97f05e4b00c1aa0"} Jul 23 12:11:41.678976 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:41.678887 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" event={"ID":"826ef539-e43c-48ac-ae5a-9545d8567801","Type":"ContainerStarted","Data":"39e4e6185895ebbef4ade8e17dbd4187e1665a48d23ac5407099b9d2a4a75743"} Jul 23 12:11:41.678976 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:41.678907 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" event={"ID":"826ef539-e43c-48ac-ae5a-9545d8567801","Type":"ContainerStarted","Data":"1d5f5855d19faad98619444b1bab329a3efb3f20858c940e3f234a4b16d07bcf"} Jul 23 12:11:41.682061 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:41.682042 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnx47" Jul 23 12:11:41.692618 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:41.692575 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" podStartSLOduration=5.123585855 podStartE2EDuration="8.692562225s" totalImagesPullingTime="3.56897637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:41.691740891 +0000 UTC m=+113.284275586" watchObservedRunningTime="2026-07-23 12:11:41.692562225 +0000 UTC m=+113.285096918" Jul 23 12:11:41.692977 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:41.692937 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2wl6p" podStartSLOduration=70.589487093 podStartE2EDuration="1m16.692927911s" totalImagesPullingTime="6.103440818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:10:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:38.711165933 +0000 UTC m=+110.303700629" watchObservedRunningTime="2026-07-23 12:11:41.692927911 +0000 UTC m=+113.285462609" Jul 23 12:11:41.743555 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:41.743469 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-644dc889b8-8d7v4" podStartSLOduration=4.167764031 podStartE2EDuration="7.743449397s" totalImagesPullingTime="3.575685366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:11:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:41.740918788 +0000 UTC m=+113.333453484" watchObservedRunningTime="2026-07-23 12:11:41.743449397 +0000 UTC m=+113.335984096" Jul 23 12:11:41.760309 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:41.760267 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wnx47" podStartSLOduration=4.911553577 podStartE2EDuration="8.760253249s" totalImagesPullingTime="3.848699672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:41.758965574 +0000 UTC m=+113.351500269" watchObservedRunningTime="2026-07-23 12:11:41.760253249 +0000 UTC m=+113.352787941" Jul 23 12:11:41.792089 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:41.792062 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:41.792222 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:41.792115 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:41.796466 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:41.796447 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:42.689265 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:42.689239 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:11:42.700238 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:42.700209 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-758588cc9d-ncpbg"] Jul 23 12:11:42.822907 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:42.822872 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:11:45.700631 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:45.700599 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3eebdce5-e954-45e3-bccd-4cd4a4aa101a","Type":"ContainerStarted","Data":"1a7f7aa7e6254cb1951754111940f67c143d2f6a2d74f07b35090d6785c2d921"} Jul 23 12:11:45.701048 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:45.700661 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3eebdce5-e954-45e3-bccd-4cd4a4aa101a","Type":"ContainerStarted","Data":"59ac824a9d7cf3f121394f053fa047a28af07a9e8fe97292c8345a8e3360b252"} Jul 23 12:11:48.718512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:48.718474 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3eebdce5-e954-45e3-bccd-4cd4a4aa101a","Type":"ContainerStarted","Data":"20b32361d0c9da5092fe1ceaa7f51554e6787768b3b6f150430fbb9cc21bcd5d"} Jul 23 12:11:48.718978 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:48.718539 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3eebdce5-e954-45e3-bccd-4cd4a4aa101a","Type":"ContainerStarted","Data":"9707a7d42ebef1bdc6539ad0c79c42ce84141fbfab2efbb6b787d7976ee33286"} Jul 23 12:11:48.718978 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:48.718566 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3eebdce5-e954-45e3-bccd-4cd4a4aa101a","Type":"ContainerStarted","Data":"d0946dc270d6dd5a909a3da77104391fbd13dbc064c62fa22c1c9d4ae6e723d4"} Jul 23 12:11:48.718978 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:48.718592 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3eebdce5-e954-45e3-bccd-4cd4a4aa101a","Type":"ContainerStarted","Data":"860621ae8a81455d17eae02db20aab3860bdea3df1765853ab3e90ab10e0644e"} Jul 23 12:11:48.752184 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:48.752135 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.253146103 podStartE2EDuration="13.75211909s" totalImagesPullingTime="9.472150878s" totalInitContainerRuntime="26.822109ms" isStatefulPod=false podCreationTimestamp="2026-07-23 12:11:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:11:48.749546841 +0000 UTC m=+120.342081533" watchObservedRunningTime="2026-07-23 12:11:48.75211909 +0000 UTC m=+120.344653783" Jul 23 12:11:49.667639 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:49.667608 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2wl6p" Jul 23 12:11:50.785488 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:50.785453 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:11:53.951753 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:53.951723 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:53.952147 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:53.951776 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:11:54.326379 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:54.326307 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:11:58.830712 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:58.830678 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs\") pod \"network-metrics-daemon-cbm6t\" (UID: \"232bfbf8-288e-4679-8bee-4c5179866c00\") " pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:11:58.833004 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:58.832984 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/232bfbf8-288e-4679-8bee-4c5179866c00-metrics-certs\") pod \"network-metrics-daemon-cbm6t\" (UID: \"232bfbf8-288e-4679-8bee-4c5179866c00\") " pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:11:58.942542 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:58.942515 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-k5z96\"" Jul 23 12:11:58.950656 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:58.950636 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cbm6t" Jul 23 12:11:59.152967 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.152943 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cbm6t"] Jul 23 12:11:59.155085 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:11:59.155058 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod232bfbf8_288e_4679_8bee_4c5179866c00.slice/crio-4ac437c3b7dbebe2a758792160703f168d8abc0d076f9dce235b1eeb50a2ead3 WatchSource:0}: Error finding container 4ac437c3b7dbebe2a758792160703f168d8abc0d076f9dce235b1eeb50a2ead3: Status 404 returned error can't find the container with id 4ac437c3b7dbebe2a758792160703f168d8abc0d076f9dce235b1eeb50a2ead3 Jul 23 12:11:59.349971 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.349915 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" podUID="c3e44bef-488e-4d15-bd18-6cbbeec2b5eb" containerName="registry" containerID="cri-o://c256f976326e096274d34708f47381e6bf24c79eac3153de0c9808fc8774016e" gracePeriod=30 Jul 23 12:11:59.610030 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.609971 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:11:59.738690 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.738662 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-trusted-ca\") pod \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " Jul 23 12:11:59.738861 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.738701 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-image-registry-private-configuration\") pod \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " Jul 23 12:11:59.738861 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.738735 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-certificates\") pod \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " Jul 23 12:11:59.738861 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.738766 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-installation-pull-secrets\") pod \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " Jul 23 12:11:59.738861 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.738796 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5b8vf\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-kube-api-access-5b8vf\") pod \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " Jul 23 12:11:59.739082 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.738964 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls\") pod \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " Jul 23 12:11:59.739082 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.739027 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-ca-trust-extracted\") pod \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " Jul 23 12:11:59.739082 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.739075 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-bound-sa-token\") pod \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\" (UID: \"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb\") " Jul 23 12:11:59.739258 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.739129 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb" (UID: "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:11:59.739258 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.739243 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb" (UID: "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:11:59.739475 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.739436 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-trusted-ca\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:11:59.739475 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.739461 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-certificates\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:11:59.741447 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.741421 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb" (UID: "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:11:59.741571 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.741473 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb" (UID: "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 12:11:59.741629 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.741563 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb" (UID: "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:11:59.741629 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.741568 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb" (UID: "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 12:11:59.741629 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.741568 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-kube-api-access-5b8vf" (OuterVolumeSpecName: "kube-api-access-5b8vf") pod "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb" (UID: "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb"). InnerVolumeSpecName "kube-api-access-5b8vf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:11:59.747323 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.747303 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb" (UID: "c3e44bef-488e-4d15-bd18-6cbbeec2b5eb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:11:59.765898 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.765875 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cbm6t" event={"ID":"232bfbf8-288e-4679-8bee-4c5179866c00","Type":"ContainerStarted","Data":"4ac437c3b7dbebe2a758792160703f168d8abc0d076f9dce235b1eeb50a2ead3"} Jul 23 12:11:59.766958 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.766935 2578 generic.go:356] "Generic (PLEG): container finished" podID="c3e44bef-488e-4d15-bd18-6cbbeec2b5eb" containerID="c256f976326e096274d34708f47381e6bf24c79eac3153de0c9808fc8774016e" exitCode=0 Jul 23 12:11:59.767053 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.766975 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" Jul 23 12:11:59.767053 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.766985 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" event={"ID":"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb","Type":"ContainerDied","Data":"c256f976326e096274d34708f47381e6bf24c79eac3153de0c9808fc8774016e"} Jul 23 12:11:59.767053 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.767013 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5999b54cbc-wg5fw" event={"ID":"c3e44bef-488e-4d15-bd18-6cbbeec2b5eb","Type":"ContainerDied","Data":"c461e3a58b48bf918048371eab3f426e5cb30f16f5e46dbfb44178af9afe86a4"} Jul 23 12:11:59.767053 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.767032 2578 scope.go:117] "RemoveContainer" containerID="c256f976326e096274d34708f47381e6bf24c79eac3153de0c9808fc8774016e" Jul 23 12:11:59.775641 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.775625 2578 scope.go:117] "RemoveContainer" containerID="c256f976326e096274d34708f47381e6bf24c79eac3153de0c9808fc8774016e" Jul 23 12:11:59.775916 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:11:59.775897 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c256f976326e096274d34708f47381e6bf24c79eac3153de0c9808fc8774016e\": container with ID starting with c256f976326e096274d34708f47381e6bf24c79eac3153de0c9808fc8774016e not found: ID does not exist" containerID="c256f976326e096274d34708f47381e6bf24c79eac3153de0c9808fc8774016e" Jul 23 12:11:59.775958 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.775925 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c256f976326e096274d34708f47381e6bf24c79eac3153de0c9808fc8774016e"} err="failed to get container status \"c256f976326e096274d34708f47381e6bf24c79eac3153de0c9808fc8774016e\": rpc error: code = NotFound desc = could not find container \"c256f976326e096274d34708f47381e6bf24c79eac3153de0c9808fc8774016e\": container with ID starting with c256f976326e096274d34708f47381e6bf24c79eac3153de0c9808fc8774016e not found: ID does not exist" Jul 23 12:11:59.791938 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.791915 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5999b54cbc-wg5fw"] Jul 23 12:11:59.796197 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.796173 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5999b54cbc-wg5fw"] Jul 23 12:11:59.840351 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.840324 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-registry-tls\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:11:59.840708 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.840356 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-ca-trust-extracted\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:11:59.840708 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.840370 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-bound-sa-token\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:11:59.840708 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.840384 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-image-registry-private-configuration\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:11:59.840708 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.840400 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-installation-pull-secrets\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:11:59.840708 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:11:59.840415 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5b8vf\" (UniqueName: \"kubernetes.io/projected/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb-kube-api-access-5b8vf\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:00.934127 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:00.934100 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3e44bef-488e-4d15-bd18-6cbbeec2b5eb" path="/var/lib/kubelet/pods/c3e44bef-488e-4d15-bd18-6cbbeec2b5eb/volumes" Jul 23 12:12:01.778329 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:01.778292 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cbm6t" event={"ID":"232bfbf8-288e-4679-8bee-4c5179866c00","Type":"ContainerStarted","Data":"6fff295655fba8eab7607316ad7615b49a3dbf858325285af4457d4ec4a06ef9"} Jul 23 12:12:01.778649 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:01.778350 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cbm6t" event={"ID":"232bfbf8-288e-4679-8bee-4c5179866c00","Type":"ContainerStarted","Data":"d9ed8b44dc6aec91f4a071f29c24d29953323c2f00d42a416a322c2a6224133b"} Jul 23 12:12:01.798380 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:01.798334 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cbm6t" podStartSLOduration=131.260399137 podStartE2EDuration="2m12.798319571s" totalImagesPullingTime="1.537920434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:09:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:12:01.795833861 +0000 UTC m=+133.388368556" watchObservedRunningTime="2026-07-23 12:12:01.798319571 +0000 UTC m=+133.390854264" Jul 23 12:12:02.653967 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:02.653902 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5fcff596c8-xlzvz" podUID="a8b43ea3-4eb8-4440-9e8b-0c08522b54b0" containerName="console" containerID="cri-o://7a94102688536c74868536ed0894a7ee392aa9dd424b2b053e46974e1401d305" gracePeriod=15 Jul 23 12:12:02.783518 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:02.783483 2578 generic.go:356] "Generic (PLEG): container finished" podID="a8b43ea3-4eb8-4440-9e8b-0c08522b54b0" containerID="7a94102688536c74868536ed0894a7ee392aa9dd424b2b053e46974e1401d305" exitCode=0 Jul 23 12:12:02.783696 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:02.783557 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fcff596c8-xlzvz" event={"ID":"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0","Type":"ContainerDied","Data":"7a94102688536c74868536ed0894a7ee392aa9dd424b2b053e46974e1401d305"} Jul 23 12:12:02.933312 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:02.933257 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:12:03.068661 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.068629 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-console-oauth-config\") pod \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " Jul 23 12:12:03.068817 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.068696 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-console-config\") pod \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " Jul 23 12:12:03.068817 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.068764 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fkdb\" (UniqueName: \"kubernetes.io/projected/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-kube-api-access-7fkdb\") pod \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " Jul 23 12:12:03.068817 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.068792 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-oauth-serving-cert\") pod \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " Jul 23 12:12:03.068988 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.068831 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-service-ca\") pod \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " Jul 23 12:12:03.068988 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.068853 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-console-serving-cert\") pod \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\" (UID: \"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0\") " Jul 23 12:12:03.069235 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.069206 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-service-ca" (OuterVolumeSpecName: "service-ca") pod "a8b43ea3-4eb8-4440-9e8b-0c08522b54b0" (UID: "a8b43ea3-4eb8-4440-9e8b-0c08522b54b0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:12:03.069402 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.069132 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-console-config" (OuterVolumeSpecName: "console-config") pod "a8b43ea3-4eb8-4440-9e8b-0c08522b54b0" (UID: "a8b43ea3-4eb8-4440-9e8b-0c08522b54b0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:12:03.069402 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.069311 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a8b43ea3-4eb8-4440-9e8b-0c08522b54b0" (UID: "a8b43ea3-4eb8-4440-9e8b-0c08522b54b0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:12:03.070980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.070949 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a8b43ea3-4eb8-4440-9e8b-0c08522b54b0" (UID: "a8b43ea3-4eb8-4440-9e8b-0c08522b54b0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 12:12:03.070980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.070972 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a8b43ea3-4eb8-4440-9e8b-0c08522b54b0" (UID: "a8b43ea3-4eb8-4440-9e8b-0c08522b54b0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 12:12:03.071107 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.071017 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-kube-api-access-7fkdb" (OuterVolumeSpecName: "kube-api-access-7fkdb") pod "a8b43ea3-4eb8-4440-9e8b-0c08522b54b0" (UID: "a8b43ea3-4eb8-4440-9e8b-0c08522b54b0"). InnerVolumeSpecName "kube-api-access-7fkdb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:12:03.169519 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.169493 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-console-oauth-config\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:03.169519 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.169514 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-console-config\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:03.169643 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.169523 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fkdb\" (UniqueName: \"kubernetes.io/projected/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-kube-api-access-7fkdb\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:03.169643 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.169532 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-oauth-serving-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:03.169643 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.169541 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-service-ca\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:03.169643 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.169549 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0-console-serving-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:03.789088 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.789058 2578 generic.go:356] "Generic (PLEG): container finished" podID="fafa146f-b22a-49a7-b7c6-263e59d7d9ab" containerID="1ef3107853cac33d1d4134fe0568facaf182ceac5f1e91593eb9953dbf7bd4ab" exitCode=0 Jul 23 12:12:03.789471 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.789129 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cns49" event={"ID":"fafa146f-b22a-49a7-b7c6-263e59d7d9ab","Type":"ContainerDied","Data":"1ef3107853cac33d1d4134fe0568facaf182ceac5f1e91593eb9953dbf7bd4ab"} Jul 23 12:12:03.789539 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.789527 2578 scope.go:117] "RemoveContainer" containerID="1ef3107853cac33d1d4134fe0568facaf182ceac5f1e91593eb9953dbf7bd4ab" Jul 23 12:12:03.790626 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.790591 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fcff596c8-xlzvz" event={"ID":"a8b43ea3-4eb8-4440-9e8b-0c08522b54b0","Type":"ContainerDied","Data":"be974e9045b5b92746091fa9a3b8bff82d719ef4183254f890b19deff25e7b78"} Jul 23 12:12:03.790626 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.790599 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fcff596c8-xlzvz" Jul 23 12:12:03.790761 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.790636 2578 scope.go:117] "RemoveContainer" containerID="7a94102688536c74868536ed0894a7ee392aa9dd424b2b053e46974e1401d305" Jul 23 12:12:03.823435 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.823407 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fcff596c8-xlzvz"] Jul 23 12:12:03.830539 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:03.830495 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5fcff596c8-xlzvz"] Jul 23 12:12:04.798935 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:04.798903 2578 generic.go:356] "Generic (PLEG): container finished" podID="5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf" containerID="791d7d63f4635839a4267442252167a563b57723330d3a77a92b525a416e6cbd" exitCode=0 Jul 23 12:12:04.799377 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:04.798976 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fwbjz" event={"ID":"5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf","Type":"ContainerDied","Data":"791d7d63f4635839a4267442252167a563b57723330d3a77a92b525a416e6cbd"} Jul 23 12:12:04.799456 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:04.799388 2578 scope.go:117] "RemoveContainer" containerID="791d7d63f4635839a4267442252167a563b57723330d3a77a92b525a416e6cbd" Jul 23 12:12:04.800950 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:04.800928 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-cns49" event={"ID":"fafa146f-b22a-49a7-b7c6-263e59d7d9ab","Type":"ContainerStarted","Data":"767704411f285ca91d2d6d51a3fef5f901bf13aa7c0cc177ef9bc45545ea3ea3"} Jul 23 12:12:04.934777 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:04.934745 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8b43ea3-4eb8-4440-9e8b-0c08522b54b0" path="/var/lib/kubelet/pods/a8b43ea3-4eb8-4440-9e8b-0c08522b54b0/volumes" Jul 23 12:12:05.808340 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:05.808308 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fwbjz" event={"ID":"5c791e2d-cdef-4dd1-8fb9-e0dcc8243bbf","Type":"ContainerStarted","Data":"33f117793d549c0dd36d9bec9854b9a51f4f08889271f6072f19ae2c91ba6a15"} Jul 23 12:12:09.715943 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:09.715901 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-758588cc9d-ncpbg" podUID="d147d50d-300b-4f7c-986f-06880311e014" containerName="console" containerID="cri-o://5922c80dc4f628c1a388d04ce2be759db1238472ba12036c48ff5da88ff9492c" gracePeriod=15 Jul 23 12:12:09.978911 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:09.978844 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:12:10.023769 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.023735 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-console-config\") pod \"d147d50d-300b-4f7c-986f-06880311e014\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " Jul 23 12:12:10.023941 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.023785 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d147d50d-300b-4f7c-986f-06880311e014-console-serving-cert\") pod \"d147d50d-300b-4f7c-986f-06880311e014\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " Jul 23 12:12:10.023941 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.023812 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-oauth-serving-cert\") pod \"d147d50d-300b-4f7c-986f-06880311e014\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " Jul 23 12:12:10.023941 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.023839 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d147d50d-300b-4f7c-986f-06880311e014-console-oauth-config\") pod \"d147d50d-300b-4f7c-986f-06880311e014\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " Jul 23 12:12:10.023941 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.023876 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjhkt\" (UniqueName: \"kubernetes.io/projected/d147d50d-300b-4f7c-986f-06880311e014-kube-api-access-gjhkt\") pod \"d147d50d-300b-4f7c-986f-06880311e014\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " Jul 23 12:12:10.023941 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.023921 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-service-ca\") pod \"d147d50d-300b-4f7c-986f-06880311e014\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " Jul 23 12:12:10.024108 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.023953 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-trusted-ca-bundle\") pod \"d147d50d-300b-4f7c-986f-06880311e014\" (UID: \"d147d50d-300b-4f7c-986f-06880311e014\") " Jul 23 12:12:10.024234 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.024179 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-console-config" (OuterVolumeSpecName: "console-config") pod "d147d50d-300b-4f7c-986f-06880311e014" (UID: "d147d50d-300b-4f7c-986f-06880311e014"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:12:10.024304 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.024281 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d147d50d-300b-4f7c-986f-06880311e014" (UID: "d147d50d-300b-4f7c-986f-06880311e014"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:12:10.024549 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.024528 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d147d50d-300b-4f7c-986f-06880311e014" (UID: "d147d50d-300b-4f7c-986f-06880311e014"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:12:10.024653 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.024622 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-service-ca" (OuterVolumeSpecName: "service-ca") pod "d147d50d-300b-4f7c-986f-06880311e014" (UID: "d147d50d-300b-4f7c-986f-06880311e014"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:12:10.026089 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.026055 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d147d50d-300b-4f7c-986f-06880311e014-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d147d50d-300b-4f7c-986f-06880311e014" (UID: "d147d50d-300b-4f7c-986f-06880311e014"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 12:12:10.026243 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.026175 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d147d50d-300b-4f7c-986f-06880311e014-kube-api-access-gjhkt" (OuterVolumeSpecName: "kube-api-access-gjhkt") pod "d147d50d-300b-4f7c-986f-06880311e014" (UID: "d147d50d-300b-4f7c-986f-06880311e014"). InnerVolumeSpecName "kube-api-access-gjhkt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:12:10.026413 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.026394 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d147d50d-300b-4f7c-986f-06880311e014-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d147d50d-300b-4f7c-986f-06880311e014" (UID: "d147d50d-300b-4f7c-986f-06880311e014"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 12:12:10.124614 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.124577 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d147d50d-300b-4f7c-986f-06880311e014-console-oauth-config\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:10.124614 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.124609 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjhkt\" (UniqueName: \"kubernetes.io/projected/d147d50d-300b-4f7c-986f-06880311e014-kube-api-access-gjhkt\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:10.124614 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.124619 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-service-ca\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:10.124815 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.124630 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-trusted-ca-bundle\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:10.124815 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.124639 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-console-config\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:10.124815 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.124647 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d147d50d-300b-4f7c-986f-06880311e014-console-serving-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:10.124815 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.124655 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d147d50d-300b-4f7c-986f-06880311e014-oauth-serving-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:10.831398 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.831365 2578 generic.go:356] "Generic (PLEG): container finished" podID="d147d50d-300b-4f7c-986f-06880311e014" containerID="5922c80dc4f628c1a388d04ce2be759db1238472ba12036c48ff5da88ff9492c" exitCode=0 Jul 23 12:12:10.831830 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.831405 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-758588cc9d-ncpbg" Jul 23 12:12:10.831830 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.831414 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-758588cc9d-ncpbg" event={"ID":"d147d50d-300b-4f7c-986f-06880311e014","Type":"ContainerDied","Data":"5922c80dc4f628c1a388d04ce2be759db1238472ba12036c48ff5da88ff9492c"} Jul 23 12:12:10.831830 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.831452 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-758588cc9d-ncpbg" event={"ID":"d147d50d-300b-4f7c-986f-06880311e014","Type":"ContainerDied","Data":"1c0df5bf5d63bda8ec8a8d74cdff8cc42f25155b2bc1d95f27c5446ae39f18d6"} Jul 23 12:12:10.831830 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.831471 2578 scope.go:117] "RemoveContainer" containerID="5922c80dc4f628c1a388d04ce2be759db1238472ba12036c48ff5da88ff9492c" Jul 23 12:12:10.845826 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.845797 2578 scope.go:117] "RemoveContainer" containerID="5922c80dc4f628c1a388d04ce2be759db1238472ba12036c48ff5da88ff9492c" Jul 23 12:12:10.846118 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:12:10.846097 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5922c80dc4f628c1a388d04ce2be759db1238472ba12036c48ff5da88ff9492c\": container with ID starting with 5922c80dc4f628c1a388d04ce2be759db1238472ba12036c48ff5da88ff9492c not found: ID does not exist" containerID="5922c80dc4f628c1a388d04ce2be759db1238472ba12036c48ff5da88ff9492c" Jul 23 12:12:10.846179 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.846127 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5922c80dc4f628c1a388d04ce2be759db1238472ba12036c48ff5da88ff9492c"} err="failed to get container status \"5922c80dc4f628c1a388d04ce2be759db1238472ba12036c48ff5da88ff9492c\": rpc error: code = NotFound desc = could not find container \"5922c80dc4f628c1a388d04ce2be759db1238472ba12036c48ff5da88ff9492c\": container with ID starting with 5922c80dc4f628c1a388d04ce2be759db1238472ba12036c48ff5da88ff9492c not found: ID does not exist" Jul 23 12:12:10.855336 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.855312 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-758588cc9d-ncpbg"] Jul 23 12:12:10.862904 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.862879 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-758588cc9d-ncpbg"] Jul 23 12:12:10.934284 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:10.934244 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d147d50d-300b-4f7c-986f-06880311e014" path="/var/lib/kubelet/pods/d147d50d-300b-4f7c-986f-06880311e014/volumes" Jul 23 12:12:13.847709 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:13.847621 2578 generic.go:356] "Generic (PLEG): container finished" podID="4eaee556-6e0b-46ae-9af7-da2785b3e34a" containerID="59134bd7d25d1e170291be25e0ba428fff0e2e2e7fdad052b239604521a7ab74" exitCode=0 Jul 23 12:12:13.847709 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:13.847691 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-8nqgf" event={"ID":"4eaee556-6e0b-46ae-9af7-da2785b3e34a","Type":"ContainerDied","Data":"59134bd7d25d1e170291be25e0ba428fff0e2e2e7fdad052b239604521a7ab74"} Jul 23 12:12:13.848142 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:13.848079 2578 scope.go:117] "RemoveContainer" containerID="59134bd7d25d1e170291be25e0ba428fff0e2e2e7fdad052b239604521a7ab74" Jul 23 12:12:13.956975 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:13.956947 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:12:13.960891 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:13.960845 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-749c994bb4-ppwqt" Jul 23 12:12:14.853940 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:14.853903 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-8nqgf" event={"ID":"4eaee556-6e0b-46ae-9af7-da2785b3e34a","Type":"ContainerStarted","Data":"d5e44ed37bb954380a6240d91d4c4111440546cd9b249ab1ecad8c3bfe3465cf"} Jul 23 12:12:35.785509 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:35.785449 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:35.805231 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:35.805170 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:35.959848 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:35.959820 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:53.901583 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:53.901548 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 12:12:53.902080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:53.901857 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="prometheus" containerID="cri-o://59ac824a9d7cf3f121394f053fa047a28af07a9e8fe97292c8345a8e3360b252" gracePeriod=600 Jul 23 12:12:53.902080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:53.901888 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="kube-rbac-proxy" containerID="cri-o://9707a7d42ebef1bdc6539ad0c79c42ce84141fbfab2efbb6b787d7976ee33286" gracePeriod=600 Jul 23 12:12:53.902080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:53.901915 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="kube-rbac-proxy-web" containerID="cri-o://d0946dc270d6dd5a909a3da77104391fbd13dbc064c62fa22c1c9d4ae6e723d4" gracePeriod=600 Jul 23 12:12:53.902080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:53.901940 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="config-reloader" containerID="cri-o://1a7f7aa7e6254cb1951754111940f67c143d2f6a2d74f07b35090d6785c2d921" gracePeriod=600 Jul 23 12:12:53.902080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:53.901888 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="thanos-sidecar" containerID="cri-o://860621ae8a81455d17eae02db20aab3860bdea3df1765853ab3e90ab10e0644e" gracePeriod=600 Jul 23 12:12:53.902080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:53.901940 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://20b32361d0c9da5092fe1ceaa7f51554e6787768b3b6f150430fbb9cc21bcd5d" gracePeriod=600 Jul 23 12:12:54.029915 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.029886 2578 generic.go:356] "Generic (PLEG): container finished" podID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerID="20b32361d0c9da5092fe1ceaa7f51554e6787768b3b6f150430fbb9cc21bcd5d" exitCode=0 Jul 23 12:12:54.029915 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.029910 2578 generic.go:356] "Generic (PLEG): container finished" podID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerID="9707a7d42ebef1bdc6539ad0c79c42ce84141fbfab2efbb6b787d7976ee33286" exitCode=0 Jul 23 12:12:54.029915 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.029916 2578 generic.go:356] "Generic (PLEG): container finished" podID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerID="d0946dc270d6dd5a909a3da77104391fbd13dbc064c62fa22c1c9d4ae6e723d4" exitCode=0 Jul 23 12:12:54.029915 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.029922 2578 generic.go:356] "Generic (PLEG): container finished" podID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerID="860621ae8a81455d17eae02db20aab3860bdea3df1765853ab3e90ab10e0644e" exitCode=0 Jul 23 12:12:54.030147 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.029927 2578 generic.go:356] "Generic (PLEG): container finished" podID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerID="1a7f7aa7e6254cb1951754111940f67c143d2f6a2d74f07b35090d6785c2d921" exitCode=0 Jul 23 12:12:54.030147 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.029932 2578 generic.go:356] "Generic (PLEG): container finished" podID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerID="59ac824a9d7cf3f121394f053fa047a28af07a9e8fe97292c8345a8e3360b252" exitCode=0 Jul 23 12:12:54.030147 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.029959 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3eebdce5-e954-45e3-bccd-4cd4a4aa101a","Type":"ContainerDied","Data":"20b32361d0c9da5092fe1ceaa7f51554e6787768b3b6f150430fbb9cc21bcd5d"} Jul 23 12:12:54.030147 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.030039 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3eebdce5-e954-45e3-bccd-4cd4a4aa101a","Type":"ContainerDied","Data":"9707a7d42ebef1bdc6539ad0c79c42ce84141fbfab2efbb6b787d7976ee33286"} Jul 23 12:12:54.030147 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.030068 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3eebdce5-e954-45e3-bccd-4cd4a4aa101a","Type":"ContainerDied","Data":"d0946dc270d6dd5a909a3da77104391fbd13dbc064c62fa22c1c9d4ae6e723d4"} Jul 23 12:12:54.030147 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.030092 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3eebdce5-e954-45e3-bccd-4cd4a4aa101a","Type":"ContainerDied","Data":"860621ae8a81455d17eae02db20aab3860bdea3df1765853ab3e90ab10e0644e"} Jul 23 12:12:54.030147 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.030114 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3eebdce5-e954-45e3-bccd-4cd4a4aa101a","Type":"ContainerDied","Data":"1a7f7aa7e6254cb1951754111940f67c143d2f6a2d74f07b35090d6785c2d921"} Jul 23 12:12:54.030147 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.030149 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3eebdce5-e954-45e3-bccd-4cd4a4aa101a","Type":"ContainerDied","Data":"59ac824a9d7cf3f121394f053fa047a28af07a9e8fe97292c8345a8e3360b252"} Jul 23 12:12:54.143400 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.143377 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:54.228599 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.228511 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-db\") pod \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " Jul 23 12:12:54.228599 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.228549 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-configmap-kubelet-serving-ca-bundle\") pod \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " Jul 23 12:12:54.228599 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.228573 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " Jul 23 12:12:54.228599 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.228591 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-rulefiles-1\") pod \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " Jul 23 12:12:54.229032 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.228616 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " Jul 23 12:12:54.229032 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.228642 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-prometheus-k8s-tls\") pod \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " Jul 23 12:12:54.229032 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.228663 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-web-config\") pod \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " Jul 23 12:12:54.229032 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.228683 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-tls-assets\") pod \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " Jul 23 12:12:54.229032 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.228701 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-rulefiles-2\") pod \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " Jul 23 12:12:54.229032 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.228723 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-kube-rbac-proxy\") pod \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " Jul 23 12:12:54.229032 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.228758 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-trusted-ca-bundle\") pod \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " Jul 23 12:12:54.229032 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.228785 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rswl\" (UniqueName: \"kubernetes.io/projected/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-kube-api-access-7rswl\") pod \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " Jul 23 12:12:54.229032 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.228814 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-configmap-serving-certs-ca-bundle\") pod \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " Jul 23 12:12:54.229032 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.228841 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-config-out\") pod \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " Jul 23 12:12:54.229032 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.228878 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-grpc-tls\") pod \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " Jul 23 12:12:54.229032 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.228907 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "3eebdce5-e954-45e3-bccd-4cd4a4aa101a" (UID: "3eebdce5-e954-45e3-bccd-4cd4a4aa101a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:12:54.229032 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.228920 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-config\") pod \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " Jul 23 12:12:54.229756 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.229099 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-configmap-metrics-client-ca\") pod \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " Jul 23 12:12:54.229756 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.229145 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-rulefiles-0\") pod \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " Jul 23 12:12:54.229756 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.229177 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-thanos-prometheus-http-client-file\") pod \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " Jul 23 12:12:54.229756 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.229228 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-metrics-client-certs\") pod \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\" (UID: \"3eebdce5-e954-45e3-bccd-4cd4a4aa101a\") " Jul 23 12:12:54.229756 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.229331 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "3eebdce5-e954-45e3-bccd-4cd4a4aa101a" (UID: "3eebdce5-e954-45e3-bccd-4cd4a4aa101a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:12:54.229756 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.229538 2578 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:54.229756 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.229559 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:54.230319 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.229864 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "3eebdce5-e954-45e3-bccd-4cd4a4aa101a" (UID: "3eebdce5-e954-45e3-bccd-4cd4a4aa101a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:12:54.230319 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.229908 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "3eebdce5-e954-45e3-bccd-4cd4a4aa101a" (UID: "3eebdce5-e954-45e3-bccd-4cd4a4aa101a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:12:54.231378 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.231349 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "3eebdce5-e954-45e3-bccd-4cd4a4aa101a" (UID: "3eebdce5-e954-45e3-bccd-4cd4a4aa101a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:12:54.231464 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.231389 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "3eebdce5-e954-45e3-bccd-4cd4a4aa101a" (UID: "3eebdce5-e954-45e3-bccd-4cd4a4aa101a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 12:12:54.231464 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.231451 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "3eebdce5-e954-45e3-bccd-4cd4a4aa101a" (UID: "3eebdce5-e954-45e3-bccd-4cd4a4aa101a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 12:12:54.231631 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.231521 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "3eebdce5-e954-45e3-bccd-4cd4a4aa101a" (UID: "3eebdce5-e954-45e3-bccd-4cd4a4aa101a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 12:12:54.231631 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.231562 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "3eebdce5-e954-45e3-bccd-4cd4a4aa101a" (UID: "3eebdce5-e954-45e3-bccd-4cd4a4aa101a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 12:12:54.231631 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.231573 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-config" (OuterVolumeSpecName: "config") pod "3eebdce5-e954-45e3-bccd-4cd4a4aa101a" (UID: "3eebdce5-e954-45e3-bccd-4cd4a4aa101a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 12:12:54.231939 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.231912 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "3eebdce5-e954-45e3-bccd-4cd4a4aa101a" (UID: "3eebdce5-e954-45e3-bccd-4cd4a4aa101a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:12:54.232040 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.232008 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "3eebdce5-e954-45e3-bccd-4cd4a4aa101a" (UID: "3eebdce5-e954-45e3-bccd-4cd4a4aa101a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:12:54.232040 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.232030 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "3eebdce5-e954-45e3-bccd-4cd4a4aa101a" (UID: "3eebdce5-e954-45e3-bccd-4cd4a4aa101a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:12:54.232543 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.232520 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-kube-api-access-7rswl" (OuterVolumeSpecName: "kube-api-access-7rswl") pod "3eebdce5-e954-45e3-bccd-4cd4a4aa101a" (UID: "3eebdce5-e954-45e3-bccd-4cd4a4aa101a"). InnerVolumeSpecName "kube-api-access-7rswl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:12:54.233566 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.233543 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "3eebdce5-e954-45e3-bccd-4cd4a4aa101a" (UID: "3eebdce5-e954-45e3-bccd-4cd4a4aa101a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:12:54.233879 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.233858 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "3eebdce5-e954-45e3-bccd-4cd4a4aa101a" (UID: "3eebdce5-e954-45e3-bccd-4cd4a4aa101a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 12:12:54.233948 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.233906 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "3eebdce5-e954-45e3-bccd-4cd4a4aa101a" (UID: "3eebdce5-e954-45e3-bccd-4cd4a4aa101a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 12:12:54.234014 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.233968 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "3eebdce5-e954-45e3-bccd-4cd4a4aa101a" (UID: "3eebdce5-e954-45e3-bccd-4cd4a4aa101a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 12:12:54.234076 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.234062 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-config-out" (OuterVolumeSpecName: "config-out") pod "3eebdce5-e954-45e3-bccd-4cd4a4aa101a" (UID: "3eebdce5-e954-45e3-bccd-4cd4a4aa101a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:12:54.243831 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.243812 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-web-config" (OuterVolumeSpecName: "web-config") pod "3eebdce5-e954-45e3-bccd-4cd4a4aa101a" (UID: "3eebdce5-e954-45e3-bccd-4cd4a4aa101a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 12:12:54.330658 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.330623 2578 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-grpc-tls\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:54.330658 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.330652 2578 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-config\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:54.330658 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.330661 2578 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-configmap-metrics-client-ca\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:54.330658 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.330670 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:54.330910 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.330680 2578 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:54.330910 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.330688 2578 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-metrics-client-certs\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:54.330910 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.330696 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-db\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:54.330910 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.330705 2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:54.330910 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.330713 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:54.330910 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.330721 2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:54.330910 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.330729 2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:54.330910 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.330737 2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-web-config\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:54.330910 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.330745 2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-tls-assets\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:54.330910 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.330753 2578 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-secret-kube-rbac-proxy\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:54.330910 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.330762 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:54.330910 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.330770 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rswl\" (UniqueName: \"kubernetes.io/projected/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-kube-api-access-7rswl\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:54.330910 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.330779 2578 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:54.330910 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:54.330786 2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3eebdce5-e954-45e3-bccd-4cd4a4aa101a-config-out\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:12:55.037070 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.037044 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.037451 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.037048 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3eebdce5-e954-45e3-bccd-4cd4a4aa101a","Type":"ContainerDied","Data":"d8d700c3ce165d0c724507443c138eb9a485bcef62ab5056e4dfc0a61e3a6200"} Jul 23 12:12:55.037451 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.037204 2578 scope.go:117] "RemoveContainer" containerID="20b32361d0c9da5092fe1ceaa7f51554e6787768b3b6f150430fbb9cc21bcd5d" Jul 23 12:12:55.044921 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.044903 2578 scope.go:117] "RemoveContainer" containerID="9707a7d42ebef1bdc6539ad0c79c42ce84141fbfab2efbb6b787d7976ee33286" Jul 23 12:12:55.052158 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.052141 2578 scope.go:117] "RemoveContainer" containerID="d0946dc270d6dd5a909a3da77104391fbd13dbc064c62fa22c1c9d4ae6e723d4" Jul 23 12:12:55.058836 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.058818 2578 scope.go:117] "RemoveContainer" containerID="860621ae8a81455d17eae02db20aab3860bdea3df1765853ab3e90ab10e0644e" Jul 23 12:12:55.065782 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.065762 2578 scope.go:117] "RemoveContainer" containerID="1a7f7aa7e6254cb1951754111940f67c143d2f6a2d74f07b35090d6785c2d921" Jul 23 12:12:55.067720 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.067699 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 12:12:55.073843 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.073768 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 12:12:55.073906 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.073872 2578 scope.go:117] "RemoveContainer" containerID="59ac824a9d7cf3f121394f053fa047a28af07a9e8fe97292c8345a8e3360b252" Jul 23 12:12:55.081048 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.081032 2578 scope.go:117] "RemoveContainer" containerID="9167ccd86b3b8d648cc6cb31e81e37b727ef4311e98f7dcaa0d822e67fb6705f" Jul 23 12:12:55.128913 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.128813 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 12:12:55.129356 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129325 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8b43ea3-4eb8-4440-9e8b-0c08522b54b0" containerName="console" Jul 23 12:12:55.129356 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129355 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8b43ea3-4eb8-4440-9e8b-0c08522b54b0" containerName="console" Jul 23 12:12:55.129512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129369 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="kube-rbac-proxy-web" Jul 23 12:12:55.129512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129375 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="kube-rbac-proxy-web" Jul 23 12:12:55.129512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129385 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="init-config-reloader" Jul 23 12:12:55.129512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129395 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="init-config-reloader" Jul 23 12:12:55.129512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129405 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="thanos-sidecar" Jul 23 12:12:55.129512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129414 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="thanos-sidecar" Jul 23 12:12:55.129512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129422 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="kube-rbac-proxy" Jul 23 12:12:55.129512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129427 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="kube-rbac-proxy" Jul 23 12:12:55.129512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129437 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3e44bef-488e-4d15-bd18-6cbbeec2b5eb" containerName="registry" Jul 23 12:12:55.129512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129442 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3e44bef-488e-4d15-bd18-6cbbeec2b5eb" containerName="registry" Jul 23 12:12:55.129512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129450 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="config-reloader" Jul 23 12:12:55.129512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129455 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="config-reloader" Jul 23 12:12:55.129512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129464 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d147d50d-300b-4f7c-986f-06880311e014" containerName="console" Jul 23 12:12:55.129512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129469 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d147d50d-300b-4f7c-986f-06880311e014" containerName="console" Jul 23 12:12:55.129512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129475 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="kube-rbac-proxy-thanos" Jul 23 12:12:55.129512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129480 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="kube-rbac-proxy-thanos" Jul 23 12:12:55.129512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129490 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="prometheus" Jul 23 12:12:55.129512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129495 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="prometheus" Jul 23 12:12:55.130042 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129546 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="kube-rbac-proxy-web" Jul 23 12:12:55.130042 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129556 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3e44bef-488e-4d15-bd18-6cbbeec2b5eb" containerName="registry" Jul 23 12:12:55.130042 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129562 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="thanos-sidecar" Jul 23 12:12:55.130042 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129568 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="prometheus" Jul 23 12:12:55.130042 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129576 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8b43ea3-4eb8-4440-9e8b-0c08522b54b0" containerName="console" Jul 23 12:12:55.130042 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129581 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="kube-rbac-proxy" Jul 23 12:12:55.130042 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129585 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d147d50d-300b-4f7c-986f-06880311e014" containerName="console" Jul 23 12:12:55.130042 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129591 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="kube-rbac-proxy-thanos" Jul 23 12:12:55.130042 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.129596 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" containerName="config-reloader" Jul 23 12:12:55.133091 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.133073 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.141918 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.141901 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2peqj2isnobmm\"" Jul 23 12:12:55.142306 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.142289 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 23 12:12:55.142404 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.142303 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 23 12:12:55.142404 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.142346 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 23 12:12:55.142404 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.142375 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 23 12:12:55.142555 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.142527 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 23 12:12:55.142602 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.142558 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 23 12:12:55.142649 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.142598 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 23 12:12:55.142720 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.142706 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 23 12:12:55.142769 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.142717 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 23 12:12:55.142769 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.142761 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 23 12:12:55.142847 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.142800 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-cl5wk\"" Jul 23 12:12:55.143172 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.143156 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 23 12:12:55.148233 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.148217 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 23 12:12:55.163487 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.163470 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 23 12:12:55.165418 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.165405 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 23 12:12:55.184469 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.184411 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 12:12:55.237667 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.237631 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.237667 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.237662 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.237855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.237683 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.237855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.237702 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/67ee23cf-372a-42ce-8db7-9c7eb49d27be-config-out\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.237855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.237758 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.237855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.237796 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-config\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.237855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.237819 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.237855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.237838 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.237855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.237856 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.238094 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.237897 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/67ee23cf-372a-42ce-8db7-9c7eb49d27be-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.238094 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.237948 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.238094 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.237970 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.238094 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.238017 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.238094 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.238035 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.238094 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.238050 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/67ee23cf-372a-42ce-8db7-9c7eb49d27be-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.238094 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.238070 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.238318 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.238099 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbtbz\" (UniqueName: \"kubernetes.io/projected/67ee23cf-372a-42ce-8db7-9c7eb49d27be-kube-api-access-gbtbz\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.238318 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.238129 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.238318 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.238146 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-web-config\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.238318 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.238165 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.339419 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.339381 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.339419 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.339422 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.339642 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.339438 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/67ee23cf-372a-42ce-8db7-9c7eb49d27be-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.339642 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.339469 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.339642 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.339485 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.339642 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.339512 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.339642 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.339536 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.339642 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.339557 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/67ee23cf-372a-42ce-8db7-9c7eb49d27be-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.339642 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.339583 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.339642 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.339606 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbtbz\" (UniqueName: \"kubernetes.io/projected/67ee23cf-372a-42ce-8db7-9c7eb49d27be-kube-api-access-gbtbz\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.339642 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.339640 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.340066 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.339668 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-web-config\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.340066 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.339689 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.340066 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.339728 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.340066 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.339749 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.340066 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.339781 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.340066 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.339817 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/67ee23cf-372a-42ce-8db7-9c7eb49d27be-config-out\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.340066 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.339846 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.340066 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.339878 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-config\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.340066 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.339910 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.340519 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.340377 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.340577 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.340550 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.340710 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.340689 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.341074 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.341054 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.341074 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.341062 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/67ee23cf-372a-42ce-8db7-9c7eb49d27be-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.341231 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.341094 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.341297 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.341264 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.342534 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.342464 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.342631 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.342597 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/67ee23cf-372a-42ce-8db7-9c7eb49d27be-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.343775 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.343737 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-web-config\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.343855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.343826 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.344686 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.344656 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.344783 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.344765 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.344847 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.344819 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.345061 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.345042 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-config\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.345168 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.345146 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/67ee23cf-372a-42ce-8db7-9c7eb49d27be-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.345249 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.345207 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.345819 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.345796 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/67ee23cf-372a-42ce-8db7-9c7eb49d27be-config-out\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.346128 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.346113 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/67ee23cf-372a-42ce-8db7-9c7eb49d27be-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.350451 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.350431 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbtbz\" (UniqueName: \"kubernetes.io/projected/67ee23cf-372a-42ce-8db7-9c7eb49d27be-kube-api-access-gbtbz\") pod \"prometheus-k8s-0\" (UID: \"67ee23cf-372a-42ce-8db7-9c7eb49d27be\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.441670 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.441586 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:12:55.582541 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:55.582470 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 12:12:55.586224 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:12:55.586183 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67ee23cf_372a_42ce_8db7_9c7eb49d27be.slice/crio-f3c309007639950051b07b8bfc940039f62ae25f1aa78a57c114a9e2428977f9 WatchSource:0}: Error finding container f3c309007639950051b07b8bfc940039f62ae25f1aa78a57c114a9e2428977f9: Status 404 returned error can't find the container with id f3c309007639950051b07b8bfc940039f62ae25f1aa78a57c114a9e2428977f9 Jul 23 12:12:56.043272 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:56.043234 2578 generic.go:356] "Generic (PLEG): container finished" podID="67ee23cf-372a-42ce-8db7-9c7eb49d27be" containerID="9e64df45ec760f4c2224f3e6356c1c0d4bbb5d4e66e80b499c8681cf87eb5c00" exitCode=0 Jul 23 12:12:56.043659 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:56.043290 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67ee23cf-372a-42ce-8db7-9c7eb49d27be","Type":"ContainerDied","Data":"9e64df45ec760f4c2224f3e6356c1c0d4bbb5d4e66e80b499c8681cf87eb5c00"} Jul 23 12:12:56.043659 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:56.043356 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67ee23cf-372a-42ce-8db7-9c7eb49d27be","Type":"ContainerStarted","Data":"f3c309007639950051b07b8bfc940039f62ae25f1aa78a57c114a9e2428977f9"} Jul 23 12:12:56.934486 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:56.934452 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3eebdce5-e954-45e3-bccd-4cd4a4aa101a" path="/var/lib/kubelet/pods/3eebdce5-e954-45e3-bccd-4cd4a4aa101a/volumes" Jul 23 12:12:57.051887 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:57.051856 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67ee23cf-372a-42ce-8db7-9c7eb49d27be","Type":"ContainerStarted","Data":"155664f5cb576c6819dcb4dfd0109acb585c466322ed5f5b25ce23e0eaf97a06"} Jul 23 12:12:57.052273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:57.051919 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67ee23cf-372a-42ce-8db7-9c7eb49d27be","Type":"ContainerStarted","Data":"3cc12b79c4a3b442f8b23a4f61b199386d89433164917e384c2eb1cd1ec2c00d"} Jul 23 12:12:57.052273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:57.051947 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67ee23cf-372a-42ce-8db7-9c7eb49d27be","Type":"ContainerStarted","Data":"1caa1810fcf1e1ccd6778a0c626a3b6c12399fa78f649a36183cb78b57f5be56"} Jul 23 12:12:57.052273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:57.051971 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67ee23cf-372a-42ce-8db7-9c7eb49d27be","Type":"ContainerStarted","Data":"4cf3f04587b41fc7df7bf8f884875c230ef8ee55470a3dc7eb3f729991303b90"} Jul 23 12:12:57.052273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:57.051994 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67ee23cf-372a-42ce-8db7-9c7eb49d27be","Type":"ContainerStarted","Data":"57289e1c01671580becb6ec12e2a16c34751f225f9331574631ada2563a502f6"} Jul 23 12:12:57.052273 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:57.052017 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67ee23cf-372a-42ce-8db7-9c7eb49d27be","Type":"ContainerStarted","Data":"42cbbddf266dabe5a4144d8eb424301bb139dcec36a78bd19c7568bd40076f10"} Jul 23 12:12:57.084562 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:12:57.084500 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.062186345 podStartE2EDuration="2.084482242s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.295897ms" isStatefulPod=false podCreationTimestamp="2026-07-23 12:12:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:12:57.082632037 +0000 UTC m=+188.675166729" watchObservedRunningTime="2026-07-23 12:12:57.084482242 +0000 UTC m=+188.677016935" Jul 23 12:13:00.442661 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:13:00.442600 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:13:55.442627 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:13:55.442570 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:13:55.457990 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:13:55.457962 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:13:56.314611 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:13:56.314586 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 12:14:33.214231 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:33.214182 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kqrpf"] Jul 23 12:14:33.216862 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:33.216839 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kqrpf" Jul 23 12:14:33.219616 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:33.219593 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 23 12:14:33.220605 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:33.220589 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 23 12:14:33.220693 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:33.220595 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-2fd6s\"" Jul 23 12:14:33.230345 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:33.230321 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kqrpf"] Jul 23 12:14:33.286058 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:33.286027 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jjdj\" (UniqueName: \"kubernetes.io/projected/a9c27a26-2071-4c16-b71d-fb7f81ee61c4-kube-api-access-9jjdj\") pod \"openshift-lws-operator-fd8ccff4c-kqrpf\" (UID: \"a9c27a26-2071-4c16-b71d-fb7f81ee61c4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kqrpf" Jul 23 12:14:33.286159 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:33.286061 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9c27a26-2071-4c16-b71d-fb7f81ee61c4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kqrpf\" (UID: \"a9c27a26-2071-4c16-b71d-fb7f81ee61c4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kqrpf" Jul 23 12:14:33.386468 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:33.386433 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jjdj\" (UniqueName: \"kubernetes.io/projected/a9c27a26-2071-4c16-b71d-fb7f81ee61c4-kube-api-access-9jjdj\") pod \"openshift-lws-operator-fd8ccff4c-kqrpf\" (UID: \"a9c27a26-2071-4c16-b71d-fb7f81ee61c4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kqrpf" Jul 23 12:14:33.386613 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:33.386470 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9c27a26-2071-4c16-b71d-fb7f81ee61c4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kqrpf\" (UID: \"a9c27a26-2071-4c16-b71d-fb7f81ee61c4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kqrpf" Jul 23 12:14:33.386857 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:33.386836 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9c27a26-2071-4c16-b71d-fb7f81ee61c4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kqrpf\" (UID: \"a9c27a26-2071-4c16-b71d-fb7f81ee61c4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kqrpf" Jul 23 12:14:33.400184 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:33.400160 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jjdj\" (UniqueName: \"kubernetes.io/projected/a9c27a26-2071-4c16-b71d-fb7f81ee61c4-kube-api-access-9jjdj\") pod \"openshift-lws-operator-fd8ccff4c-kqrpf\" (UID: \"a9c27a26-2071-4c16-b71d-fb7f81ee61c4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kqrpf" Jul 23 12:14:33.541355 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:33.541288 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kqrpf" Jul 23 12:14:33.679071 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:33.679039 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kqrpf"] Jul 23 12:14:33.683141 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:14:33.683115 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9c27a26_2071_4c16_b71d_fb7f81ee61c4.slice/crio-4fd3fa51afe3a50fc61d53de1ff954f57e5b6aaf53018268c409a37cf338c041 WatchSource:0}: Error finding container 4fd3fa51afe3a50fc61d53de1ff954f57e5b6aaf53018268c409a37cf338c041: Status 404 returned error can't find the container with id 4fd3fa51afe3a50fc61d53de1ff954f57e5b6aaf53018268c409a37cf338c041 Jul 23 12:14:34.448120 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:34.448085 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kqrpf" event={"ID":"a9c27a26-2071-4c16-b71d-fb7f81ee61c4","Type":"ContainerStarted","Data":"4fd3fa51afe3a50fc61d53de1ff954f57e5b6aaf53018268c409a37cf338c041"} Jul 23 12:14:36.466163 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:36.466130 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kqrpf" event={"ID":"a9c27a26-2071-4c16-b71d-fb7f81ee61c4","Type":"ContainerStarted","Data":"92ee02da20670c59e03368af58afbb82d9aae956250e0cbe05fabc7aa948eb05"} Jul 23 12:14:36.488935 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:36.488887 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kqrpf" podStartSLOduration=0.795958809 podStartE2EDuration="3.488870352s" totalImagesPullingTime="2.692911543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:14:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:14:36.487006177 +0000 UTC m=+288.079540869" watchObservedRunningTime="2026-07-23 12:14:36.488870352 +0000 UTC m=+288.081405045" Jul 23 12:14:48.822512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:48.822479 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:14:48.826577 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:48.826557 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 23 12:14:48.827006 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:48.826989 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:14:56.442037 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.442000 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b"] Jul 23 12:14:56.445391 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.445375 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" Jul 23 12:14:56.448743 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.448690 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 23 12:14:56.448870 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.448748 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 23 12:14:56.448870 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.448772 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 23 12:14:56.448870 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.448815 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4zrkc\"" Jul 23 12:14:56.449203 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.449172 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 23 12:14:56.464356 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.464327 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b"] Jul 23 12:14:56.581968 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.581938 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcb66\" (UniqueName: \"kubernetes.io/projected/1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8-kube-api-access-wcb66\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b\" (UID: \"1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" Jul 23 12:14:56.582108 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.581979 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b\" (UID: \"1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" Jul 23 12:14:56.582108 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.582052 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8-manifests\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b\" (UID: \"1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" Jul 23 12:14:56.582108 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.582082 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b\" (UID: \"1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" Jul 23 12:14:56.582108 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.582105 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8-tmp\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b\" (UID: \"1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" Jul 23 12:14:56.683225 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.683173 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcb66\" (UniqueName: \"kubernetes.io/projected/1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8-kube-api-access-wcb66\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b\" (UID: \"1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" Jul 23 12:14:56.683375 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.683246 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b\" (UID: \"1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" Jul 23 12:14:56.683375 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.683312 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8-manifests\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b\" (UID: \"1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" Jul 23 12:14:56.683375 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.683340 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b\" (UID: \"1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" Jul 23 12:14:56.683493 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.683372 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8-tmp\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b\" (UID: \"1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" Jul 23 12:14:56.683785 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.683755 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8-manifests\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b\" (UID: \"1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" Jul 23 12:14:56.685682 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.685659 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8-tmp\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b\" (UID: \"1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" Jul 23 12:14:56.685758 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.685733 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b\" (UID: \"1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" Jul 23 12:14:56.685881 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.685865 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b\" (UID: \"1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" Jul 23 12:14:56.692518 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.692467 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcb66\" (UniqueName: \"kubernetes.io/projected/1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8-kube-api-access-wcb66\") pod \"opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b\" (UID: \"1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8\") " pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" Jul 23 12:14:56.755990 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.755961 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" Jul 23 12:14:56.906023 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.905994 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b"] Jul 23 12:14:56.909647 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:14:56.909621 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dc2e72c_6f6b_40ed_845b_d7f4dbc5c3a8.slice/crio-7e310a00b5c259a689821daf7c21218c87c5530e057bed8ddc9b8bd128a7921a WatchSource:0}: Error finding container 7e310a00b5c259a689821daf7c21218c87c5530e057bed8ddc9b8bd128a7921a: Status 404 returned error can't find the container with id 7e310a00b5c259a689821daf7c21218c87c5530e057bed8ddc9b8bd128a7921a Jul 23 12:14:56.911482 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:56.911461 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 12:14:57.554164 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:14:57.554132 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" event={"ID":"1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8","Type":"ContainerStarted","Data":"7e310a00b5c259a689821daf7c21218c87c5530e057bed8ddc9b8bd128a7921a"} Jul 23 12:15:00.575232 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:00.575185 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" event={"ID":"1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8","Type":"ContainerStarted","Data":"f66756e29fb552b7208bba2ef6b0ff9b9792c9e730cb79fad6f5d6fa0e038f01"} Jul 23 12:15:01.580799 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:01.580691 2578 generic.go:356] "Generic (PLEG): container finished" podID="1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8" containerID="f66756e29fb552b7208bba2ef6b0ff9b9792c9e730cb79fad6f5d6fa0e038f01" exitCode=0 Jul 23 12:15:01.580799 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:01.580750 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" event={"ID":"1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8","Type":"ContainerDied","Data":"f66756e29fb552b7208bba2ef6b0ff9b9792c9e730cb79fad6f5d6fa0e038f01"} Jul 23 12:15:02.586585 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:02.586546 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" event={"ID":"1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8","Type":"ContainerStarted","Data":"24a1dfbd182c14ad71a051684c6ef336384ba1cca7ed574c21e28b2488c3e8ec"} Jul 23 12:15:02.586981 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:02.586681 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" Jul 23 12:15:02.612309 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:02.612175 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" podStartSLOduration=1.5960137429999999 podStartE2EDuration="6.612154703s" totalImagesPullingTime="3.991358801s" totalInitContainerRuntime="1.024782159s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:14:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:15:02.60988341 +0000 UTC m=+314.202418112" watchObservedRunningTime="2026-07-23 12:15:02.612154703 +0000 UTC m=+314.204689398" Jul 23 12:15:13.595125 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:13.595096 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b" Jul 23 12:15:25.303317 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.303286 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq"] Jul 23 12:15:25.308031 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.308011 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq" Jul 23 12:15:25.310853 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.310770 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 23 12:15:25.311719 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.311701 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 23 12:15:25.311823 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.311784 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vtwwt\"" Jul 23 12:15:25.311883 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.311719 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 23 12:15:25.322380 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.322360 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq"] Jul 23 12:15:25.350656 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.350636 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a4b3135-dfbc-44a4-8753-acdd19322d4d-cert\") pod \"lws-controller-manager-7c76dc7d5f-ctjfq\" (UID: \"3a4b3135-dfbc-44a4-8753-acdd19322d4d\") " pod="openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq" Jul 23 12:15:25.350756 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.350668 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3a4b3135-dfbc-44a4-8753-acdd19322d4d-manager-config\") pod \"lws-controller-manager-7c76dc7d5f-ctjfq\" (UID: \"3a4b3135-dfbc-44a4-8753-acdd19322d4d\") " pod="openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq" Jul 23 12:15:25.350756 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.350693 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2hdd\" (UniqueName: \"kubernetes.io/projected/3a4b3135-dfbc-44a4-8753-acdd19322d4d-kube-api-access-x2hdd\") pod \"lws-controller-manager-7c76dc7d5f-ctjfq\" (UID: \"3a4b3135-dfbc-44a4-8753-acdd19322d4d\") " pod="openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq" Jul 23 12:15:25.350833 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.350757 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a4b3135-dfbc-44a4-8753-acdd19322d4d-metrics-cert\") pod \"lws-controller-manager-7c76dc7d5f-ctjfq\" (UID: \"3a4b3135-dfbc-44a4-8753-acdd19322d4d\") " pod="openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq" Jul 23 12:15:25.453472 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.451641 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a4b3135-dfbc-44a4-8753-acdd19322d4d-cert\") pod \"lws-controller-manager-7c76dc7d5f-ctjfq\" (UID: \"3a4b3135-dfbc-44a4-8753-acdd19322d4d\") " pod="openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq" Jul 23 12:15:25.453472 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.451695 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3a4b3135-dfbc-44a4-8753-acdd19322d4d-manager-config\") pod \"lws-controller-manager-7c76dc7d5f-ctjfq\" (UID: \"3a4b3135-dfbc-44a4-8753-acdd19322d4d\") " pod="openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq" Jul 23 12:15:25.453472 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.451736 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2hdd\" (UniqueName: \"kubernetes.io/projected/3a4b3135-dfbc-44a4-8753-acdd19322d4d-kube-api-access-x2hdd\") pod \"lws-controller-manager-7c76dc7d5f-ctjfq\" (UID: \"3a4b3135-dfbc-44a4-8753-acdd19322d4d\") " pod="openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq" Jul 23 12:15:25.453472 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.451776 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a4b3135-dfbc-44a4-8753-acdd19322d4d-metrics-cert\") pod \"lws-controller-manager-7c76dc7d5f-ctjfq\" (UID: \"3a4b3135-dfbc-44a4-8753-acdd19322d4d\") " pod="openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq" Jul 23 12:15:25.453472 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.453126 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3a4b3135-dfbc-44a4-8753-acdd19322d4d-manager-config\") pod \"lws-controller-manager-7c76dc7d5f-ctjfq\" (UID: \"3a4b3135-dfbc-44a4-8753-acdd19322d4d\") " pod="openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq" Jul 23 12:15:25.455138 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.455084 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a4b3135-dfbc-44a4-8753-acdd19322d4d-metrics-cert\") pod \"lws-controller-manager-7c76dc7d5f-ctjfq\" (UID: \"3a4b3135-dfbc-44a4-8753-acdd19322d4d\") " pod="openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq" Jul 23 12:15:25.457238 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.457213 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a4b3135-dfbc-44a4-8753-acdd19322d4d-cert\") pod \"lws-controller-manager-7c76dc7d5f-ctjfq\" (UID: \"3a4b3135-dfbc-44a4-8753-acdd19322d4d\") " pod="openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq" Jul 23 12:15:25.465671 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.465629 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2hdd\" (UniqueName: \"kubernetes.io/projected/3a4b3135-dfbc-44a4-8753-acdd19322d4d-kube-api-access-x2hdd\") pod \"lws-controller-manager-7c76dc7d5f-ctjfq\" (UID: \"3a4b3135-dfbc-44a4-8753-acdd19322d4d\") " pod="openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq" Jul 23 12:15:25.620314 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.620286 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq" Jul 23 12:15:25.765665 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:25.765632 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq"] Jul 23 12:15:25.768727 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:15:25.768692 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a4b3135_dfbc_44a4_8753_acdd19322d4d.slice/crio-c0a70e5aa53dd6bc8f910a832509f2d16a07aeaa6f46ced3314c17e769dd6438 WatchSource:0}: Error finding container c0a70e5aa53dd6bc8f910a832509f2d16a07aeaa6f46ced3314c17e769dd6438: Status 404 returned error can't find the container with id c0a70e5aa53dd6bc8f910a832509f2d16a07aeaa6f46ced3314c17e769dd6438 Jul 23 12:15:26.706696 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:26.706663 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq" event={"ID":"3a4b3135-dfbc-44a4-8753-acdd19322d4d","Type":"ContainerStarted","Data":"c0a70e5aa53dd6bc8f910a832509f2d16a07aeaa6f46ced3314c17e769dd6438"} Jul 23 12:15:26.984180 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:26.984099 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9"] Jul 23 12:15:26.988818 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:26.988802 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.000665 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.000644 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 23 12:15:27.001008 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.000990 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 23 12:15:27.001106 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.001041 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 23 12:15:27.001172 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.001132 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 23 12:15:27.001274 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.001259 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 23 12:15:27.001504 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.001489 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-z5gd8\"" Jul 23 12:15:27.001725 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.001712 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 23 12:15:27.066659 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.066632 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/219312fa-7eeb-4989-9032-2da34674f1e4-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.066796 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.066664 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/219312fa-7eeb-4989-9032-2da34674f1e4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.066796 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.066717 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/219312fa-7eeb-4989-9032-2da34674f1e4-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.066796 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.066741 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8sdq\" (UniqueName: \"kubernetes.io/projected/219312fa-7eeb-4989-9032-2da34674f1e4-kube-api-access-l8sdq\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.066796 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.066768 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/219312fa-7eeb-4989-9032-2da34674f1e4-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.066796 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.066790 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/219312fa-7eeb-4989-9032-2da34674f1e4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.067041 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.066815 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/219312fa-7eeb-4989-9032-2da34674f1e4-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.131572 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.131543 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9"] Jul 23 12:15:27.167819 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.167785 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/219312fa-7eeb-4989-9032-2da34674f1e4-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.167819 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.167821 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/219312fa-7eeb-4989-9032-2da34674f1e4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.168032 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.167848 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/219312fa-7eeb-4989-9032-2da34674f1e4-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.168032 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.167886 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/219312fa-7eeb-4989-9032-2da34674f1e4-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.168032 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.167909 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/219312fa-7eeb-4989-9032-2da34674f1e4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.168032 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.167964 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/219312fa-7eeb-4989-9032-2da34674f1e4-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.168032 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.167983 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8sdq\" (UniqueName: \"kubernetes.io/projected/219312fa-7eeb-4989-9032-2da34674f1e4-kube-api-access-l8sdq\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.168833 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.168798 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/219312fa-7eeb-4989-9032-2da34674f1e4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.170360 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.170336 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/219312fa-7eeb-4989-9032-2da34674f1e4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.170466 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.170409 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/219312fa-7eeb-4989-9032-2da34674f1e4-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.170466 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.170428 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/219312fa-7eeb-4989-9032-2da34674f1e4-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.170533 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.170507 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/219312fa-7eeb-4989-9032-2da34674f1e4-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.185682 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.185653 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8sdq\" (UniqueName: \"kubernetes.io/projected/219312fa-7eeb-4989-9032-2da34674f1e4-kube-api-access-l8sdq\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.186252 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.186226 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/219312fa-7eeb-4989-9032-2da34674f1e4-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-hzjq9\" (UID: \"219312fa-7eeb-4989-9032-2da34674f1e4\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.297724 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.297640 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:27.435558 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.435534 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9"] Jul 23 12:15:27.436885 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:15:27.436864 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod219312fa_7eeb_4989_9032_2da34674f1e4.slice/crio-a707db56c4c1c2286a27fb49fe04be6681bada6a1851b20f0923ead413f738e0 WatchSource:0}: Error finding container a707db56c4c1c2286a27fb49fe04be6681bada6a1851b20f0923ead413f738e0: Status 404 returned error can't find the container with id a707db56c4c1c2286a27fb49fe04be6681bada6a1851b20f0923ead413f738e0 Jul 23 12:15:27.712267 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:27.712235 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" event={"ID":"219312fa-7eeb-4989-9032-2da34674f1e4","Type":"ContainerStarted","Data":"a707db56c4c1c2286a27fb49fe04be6681bada6a1851b20f0923ead413f738e0"} Jul 23 12:15:30.720707 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:30.720672 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:15:30.721014 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:30.720748 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:15:31.738762 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:31.738726 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq" event={"ID":"3a4b3135-dfbc-44a4-8753-acdd19322d4d","Type":"ContainerStarted","Data":"083245841eef3a7cfc691138d6651b17dcf92434d5b542949c152c5a1a6ddde9"} Jul 23 12:15:31.739342 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:31.738785 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq" Jul 23 12:15:31.740276 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:31.740249 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" event={"ID":"219312fa-7eeb-4989-9032-2da34674f1e4","Type":"ContainerStarted","Data":"1819958d26bb7e28103cbbd08aa2707aa0338df9a9fe494b633cf11ebcb81563"} Jul 23 12:15:31.740489 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:31.740469 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:31.742044 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:31.742026 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" Jul 23 12:15:31.801428 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:31.801357 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq" podStartSLOduration=1.852629597 podStartE2EDuration="6.801336764s" totalImagesPullingTime="4.948707167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:15:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:15:31.799497147 +0000 UTC m=+343.392031836" watchObservedRunningTime="2026-07-23 12:15:31.801336764 +0000 UTC m=+343.393871458" Jul 23 12:15:31.833483 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:31.833426 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hzjq9" podStartSLOduration=2.551909597 podStartE2EDuration="5.833403723s" totalImagesPullingTime="3.281494126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:15:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:15:31.831911068 +0000 UTC m=+343.424445761" watchObservedRunningTime="2026-07-23 12:15:31.833403723 +0000 UTC m=+343.425938422" Jul 23 12:15:42.748386 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:15:42.748310 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7c76dc7d5f-ctjfq" Jul 23 12:16:48.466132 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.466094 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d76c58cfd-7ncbt"] Jul 23 12:16:48.468327 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.468310 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.471465 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.471446 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 23 12:16:48.472214 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.472182 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 23 12:16:48.473598 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.473579 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 23 12:16:48.473697 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.473579 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 23 12:16:48.473911 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.473867 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 23 12:16:48.474013 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.473994 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-6gcs6\"" Jul 23 12:16:48.479114 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.479094 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 23 12:16:48.483759 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.483735 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d76c58cfd-7ncbt"] Jul 23 12:16:48.599472 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.599438 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4ecfe4d-2076-49e4-b478-bf544b78c175-trusted-ca-bundle\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.599472 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.599487 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4ecfe4d-2076-49e4-b478-bf544b78c175-console-serving-cert\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.599694 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.599556 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b4ecfe4d-2076-49e4-b478-bf544b78c175-console-config\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.599694 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.599577 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnwhl\" (UniqueName: \"kubernetes.io/projected/b4ecfe4d-2076-49e4-b478-bf544b78c175-kube-api-access-lnwhl\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.599694 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.599652 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b4ecfe4d-2076-49e4-b478-bf544b78c175-oauth-serving-cert\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.599694 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.599691 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b4ecfe4d-2076-49e4-b478-bf544b78c175-console-oauth-config\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.599821 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.599713 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b4ecfe4d-2076-49e4-b478-bf544b78c175-service-ca\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.700182 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.700145 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b4ecfe4d-2076-49e4-b478-bf544b78c175-console-config\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.700182 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.700181 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnwhl\" (UniqueName: \"kubernetes.io/projected/b4ecfe4d-2076-49e4-b478-bf544b78c175-kube-api-access-lnwhl\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.700466 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.700242 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b4ecfe4d-2076-49e4-b478-bf544b78c175-oauth-serving-cert\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.700466 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.700264 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b4ecfe4d-2076-49e4-b478-bf544b78c175-console-oauth-config\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.700466 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.700282 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b4ecfe4d-2076-49e4-b478-bf544b78c175-service-ca\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.700466 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.700317 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4ecfe4d-2076-49e4-b478-bf544b78c175-trusted-ca-bundle\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.700466 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.700348 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4ecfe4d-2076-49e4-b478-bf544b78c175-console-serving-cert\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.703134 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.701575 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b4ecfe4d-2076-49e4-b478-bf544b78c175-service-ca\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.703134 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.701603 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b4ecfe4d-2076-49e4-b478-bf544b78c175-oauth-serving-cert\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.703134 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.701918 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4ecfe4d-2076-49e4-b478-bf544b78c175-trusted-ca-bundle\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.707563 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.707537 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b4ecfe4d-2076-49e4-b478-bf544b78c175-console-oauth-config\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.707723 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.707698 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4ecfe4d-2076-49e4-b478-bf544b78c175-console-serving-cert\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.708170 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.708151 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b4ecfe4d-2076-49e4-b478-bf544b78c175-console-config\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.710836 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.710816 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnwhl\" (UniqueName: \"kubernetes.io/projected/b4ecfe4d-2076-49e4-b478-bf544b78c175-kube-api-access-lnwhl\") pod \"console-7d76c58cfd-7ncbt\" (UID: \"b4ecfe4d-2076-49e4-b478-bf544b78c175\") " pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.777666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.777580 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:48.921809 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:48.921786 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d76c58cfd-7ncbt"] Jul 23 12:16:48.924479 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:16:48.924450 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4ecfe4d_2076_49e4_b478_bf544b78c175.slice/crio-8dd8fa6063e88a1dad69dd6c15bef74c7362f5e8c4815b41fbd02e6a590c5728 WatchSource:0}: Error finding container 8dd8fa6063e88a1dad69dd6c15bef74c7362f5e8c4815b41fbd02e6a590c5728: Status 404 returned error can't find the container with id 8dd8fa6063e88a1dad69dd6c15bef74c7362f5e8c4815b41fbd02e6a590c5728 Jul 23 12:16:49.099567 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:49.099479 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d76c58cfd-7ncbt" event={"ID":"b4ecfe4d-2076-49e4-b478-bf544b78c175","Type":"ContainerStarted","Data":"2c3a2194c7cf69845c80c552b75a0755797ea9e3a0418a2a4687edddcb98a1f9"} Jul 23 12:16:49.099567 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:49.099535 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d76c58cfd-7ncbt" event={"ID":"b4ecfe4d-2076-49e4-b478-bf544b78c175","Type":"ContainerStarted","Data":"8dd8fa6063e88a1dad69dd6c15bef74c7362f5e8c4815b41fbd02e6a590c5728"} Jul 23 12:16:49.131274 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:49.131216 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d76c58cfd-7ncbt" podStartSLOduration=1.131182035 podStartE2EDuration="1.131182035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:16:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:16:49.13115616 +0000 UTC m=+420.723690852" watchObservedRunningTime="2026-07-23 12:16:49.131182035 +0000 UTC m=+420.723716727" Jul 23 12:16:58.778514 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:58.778465 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:58.778952 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:58.778544 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:58.783388 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:58.783363 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:16:59.155022 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:16:59.154992 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d76c58cfd-7ncbt" Jul 23 12:17:14.046569 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.046487 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w"] Jul 23 12:17:14.048885 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.048868 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w" Jul 23 12:17:14.052936 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.052916 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 23 12:17:14.053209 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.053163 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 23 12:17:14.053473 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.053458 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5tb7f\"" Jul 23 12:17:14.079641 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.079610 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w"] Jul 23 12:17:14.139309 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.139271 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f8f63e7-be2b-4242-ad70-791d25ee2448-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-md96w\" (UID: \"4f8f63e7-be2b-4242-ad70-791d25ee2448\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w" Jul 23 12:17:14.139485 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.139382 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4f7n\" (UniqueName: \"kubernetes.io/projected/4f8f63e7-be2b-4242-ad70-791d25ee2448-kube-api-access-g4f7n\") pod \"kuadrant-operator-controller-manager-55467d9f7-md96w\" (UID: \"4f8f63e7-be2b-4242-ad70-791d25ee2448\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w" Jul 23 12:17:14.240855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.240813 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f8f63e7-be2b-4242-ad70-791d25ee2448-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-md96w\" (UID: \"4f8f63e7-be2b-4242-ad70-791d25ee2448\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w" Jul 23 12:17:14.241035 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.240888 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4f7n\" (UniqueName: \"kubernetes.io/projected/4f8f63e7-be2b-4242-ad70-791d25ee2448-kube-api-access-g4f7n\") pod \"kuadrant-operator-controller-manager-55467d9f7-md96w\" (UID: \"4f8f63e7-be2b-4242-ad70-791d25ee2448\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w" Jul 23 12:17:14.241264 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.241238 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f8f63e7-be2b-4242-ad70-791d25ee2448-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-md96w\" (UID: \"4f8f63e7-be2b-4242-ad70-791d25ee2448\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w" Jul 23 12:17:14.251641 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.251618 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4f7n\" (UniqueName: \"kubernetes.io/projected/4f8f63e7-be2b-4242-ad70-791d25ee2448-kube-api-access-g4f7n\") pod \"kuadrant-operator-controller-manager-55467d9f7-md96w\" (UID: \"4f8f63e7-be2b-4242-ad70-791d25ee2448\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w" Jul 23 12:17:14.359478 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.359444 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w" Jul 23 12:17:14.499151 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.499077 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w"] Jul 23 12:17:14.501770 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:17:14.501730 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f8f63e7_be2b_4242_ad70_791d25ee2448.slice/crio-113e27572c4dbccb8dc3664b692a9e4985276a1ec948bdcd2fc266338999ea83 WatchSource:0}: Error finding container 113e27572c4dbccb8dc3664b692a9e4985276a1ec948bdcd2fc266338999ea83: Status 404 returned error can't find the container with id 113e27572c4dbccb8dc3664b692a9e4985276a1ec948bdcd2fc266338999ea83 Jul 23 12:17:14.519139 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.519108 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w"] Jul 23 12:17:14.521523 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.521484 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w"] Jul 23 12:17:14.551431 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.551400 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk"] Jul 23 12:17:14.554669 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.554647 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk" Jul 23 12:17:14.570403 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.570378 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk"] Jul 23 12:17:14.644264 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.644162 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml5wf\" (UniqueName: \"kubernetes.io/projected/06536fdd-1174-43e9-872a-624917a01d6c-kube-api-access-ml5wf\") pod \"kuadrant-operator-controller-manager-55467d9f7-wzhvk\" (UID: \"06536fdd-1174-43e9-872a-624917a01d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk" Jul 23 12:17:14.644421 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.644268 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/06536fdd-1174-43e9-872a-624917a01d6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-wzhvk\" (UID: \"06536fdd-1174-43e9-872a-624917a01d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk" Jul 23 12:17:14.745436 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.745395 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml5wf\" (UniqueName: \"kubernetes.io/projected/06536fdd-1174-43e9-872a-624917a01d6c-kube-api-access-ml5wf\") pod \"kuadrant-operator-controller-manager-55467d9f7-wzhvk\" (UID: \"06536fdd-1174-43e9-872a-624917a01d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk" Jul 23 12:17:14.745613 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.745446 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/06536fdd-1174-43e9-872a-624917a01d6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-wzhvk\" (UID: \"06536fdd-1174-43e9-872a-624917a01d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk" Jul 23 12:17:14.745816 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.745794 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/06536fdd-1174-43e9-872a-624917a01d6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-wzhvk\" (UID: \"06536fdd-1174-43e9-872a-624917a01d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk" Jul 23 12:17:14.757601 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.757579 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml5wf\" (UniqueName: \"kubernetes.io/projected/06536fdd-1174-43e9-872a-624917a01d6c-kube-api-access-ml5wf\") pod \"kuadrant-operator-controller-manager-55467d9f7-wzhvk\" (UID: \"06536fdd-1174-43e9-872a-624917a01d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk" Jul 23 12:17:14.866276 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:14.866242 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk" Jul 23 12:17:15.004880 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:15.004693 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk"] Jul 23 12:17:15.007772 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:17:15.007732 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06536fdd_1174_43e9_872a_624917a01d6c.slice/crio-83e2fbe2ee6ef886334c2e4ac9e5052311a3a5cf3040fef83b26ccf77e85da86 WatchSource:0}: Error finding container 83e2fbe2ee6ef886334c2e4ac9e5052311a3a5cf3040fef83b26ccf77e85da86: Status 404 returned error can't find the container with id 83e2fbe2ee6ef886334c2e4ac9e5052311a3a5cf3040fef83b26ccf77e85da86 Jul 23 12:17:15.229626 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:15.229540 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk" event={"ID":"06536fdd-1174-43e9-872a-624917a01d6c","Type":"ContainerStarted","Data":"83e2fbe2ee6ef886334c2e4ac9e5052311a3a5cf3040fef83b26ccf77e85da86"} Jul 23 12:17:20.257810 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:20.257746 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w" podUID="4f8f63e7-be2b-4242-ad70-791d25ee2448" containerName="manager" containerID="cri-o://5d3bc6740576a6caea25571983d70f462376ad81fc51f5f34556978cda935668" gracePeriod=2 Jul 23 12:17:20.259358 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:20.259323 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk" event={"ID":"06536fdd-1174-43e9-872a-624917a01d6c","Type":"ContainerStarted","Data":"a9bae3d9ed33d6fbb060805f211a394581ac8d14badb9a3e5b338ee2e331bfd2"} Jul 23 12:17:20.259494 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:20.259465 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk" Jul 23 12:17:20.260702 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:17:20.260681 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-md96w\" is forbidden: User \"system:node:ip-10-0-138-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-37.ec2.internal' and this object" podUID="4f8f63e7-be2b-4242-ad70-791d25ee2448" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w" Jul 23 12:17:20.284620 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:20.284564 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk" podStartSLOduration=1.883270346 podStartE2EDuration="6.284543507s" totalImagesPullingTime="4.401273161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:17:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:17:20.283296075 +0000 UTC m=+451.875830767" watchObservedRunningTime="2026-07-23 12:17:20.284543507 +0000 UTC m=+451.877078310" Jul 23 12:17:20.285550 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:17:20.285509 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-md96w\" is forbidden: User \"system:node:ip-10-0-138-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-37.ec2.internal' and this object" podUID="4f8f63e7-be2b-4242-ad70-791d25ee2448" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w" Jul 23 12:17:20.502748 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:20.502718 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w" Jul 23 12:17:20.505044 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:17:20.505011 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-md96w\" is forbidden: User \"system:node:ip-10-0-138-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-37.ec2.internal' and this object" podUID="4f8f63e7-be2b-4242-ad70-791d25ee2448" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w" Jul 23 12:17:20.600636 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:20.600538 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f8f63e7-be2b-4242-ad70-791d25ee2448-extensions-socket-volume\") pod \"4f8f63e7-be2b-4242-ad70-791d25ee2448\" (UID: \"4f8f63e7-be2b-4242-ad70-791d25ee2448\") " Jul 23 12:17:20.600818 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:20.600656 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4f7n\" (UniqueName: \"kubernetes.io/projected/4f8f63e7-be2b-4242-ad70-791d25ee2448-kube-api-access-g4f7n\") pod \"4f8f63e7-be2b-4242-ad70-791d25ee2448\" (UID: \"4f8f63e7-be2b-4242-ad70-791d25ee2448\") " Jul 23 12:17:20.600895 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:20.600833 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f8f63e7-be2b-4242-ad70-791d25ee2448-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4f8f63e7-be2b-4242-ad70-791d25ee2448" (UID: "4f8f63e7-be2b-4242-ad70-791d25ee2448"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:17:20.602897 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:20.602869 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f8f63e7-be2b-4242-ad70-791d25ee2448-kube-api-access-g4f7n" (OuterVolumeSpecName: "kube-api-access-g4f7n") pod "4f8f63e7-be2b-4242-ad70-791d25ee2448" (UID: "4f8f63e7-be2b-4242-ad70-791d25ee2448"). InnerVolumeSpecName "kube-api-access-g4f7n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:17:20.701541 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:20.701498 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4f7n\" (UniqueName: \"kubernetes.io/projected/4f8f63e7-be2b-4242-ad70-791d25ee2448-kube-api-access-g4f7n\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:17:20.701541 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:20.701535 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f8f63e7-be2b-4242-ad70-791d25ee2448-extensions-socket-volume\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:17:20.935071 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:20.935039 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f8f63e7-be2b-4242-ad70-791d25ee2448" path="/var/lib/kubelet/pods/4f8f63e7-be2b-4242-ad70-791d25ee2448/volumes" Jul 23 12:17:21.265339 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:21.265250 2578 generic.go:356] "Generic (PLEG): container finished" podID="4f8f63e7-be2b-4242-ad70-791d25ee2448" containerID="5d3bc6740576a6caea25571983d70f462376ad81fc51f5f34556978cda935668" exitCode=2 Jul 23 12:17:21.265339 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:21.265296 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w" Jul 23 12:17:21.265792 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:21.265351 2578 scope.go:117] "RemoveContainer" containerID="5d3bc6740576a6caea25571983d70f462376ad81fc51f5f34556978cda935668" Jul 23 12:17:21.267785 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:17:21.267757 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-md96w\" is forbidden: User \"system:node:ip-10-0-138-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-37.ec2.internal' and this object" podUID="4f8f63e7-be2b-4242-ad70-791d25ee2448" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w" Jul 23 12:17:21.269666 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:17:21.269638 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-md96w\" is forbidden: User \"system:node:ip-10-0-138-37.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-37.ec2.internal' and this object" podUID="4f8f63e7-be2b-4242-ad70-791d25ee2448" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-md96w" Jul 23 12:17:21.273899 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:21.273871 2578 scope.go:117] "RemoveContainer" containerID="5d3bc6740576a6caea25571983d70f462376ad81fc51f5f34556978cda935668" Jul 23 12:17:21.274182 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:17:21.274152 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d3bc6740576a6caea25571983d70f462376ad81fc51f5f34556978cda935668\": container with ID starting with 5d3bc6740576a6caea25571983d70f462376ad81fc51f5f34556978cda935668 not found: ID does not exist" containerID="5d3bc6740576a6caea25571983d70f462376ad81fc51f5f34556978cda935668" Jul 23 12:17:21.274295 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:21.274188 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d3bc6740576a6caea25571983d70f462376ad81fc51f5f34556978cda935668"} err="failed to get container status \"5d3bc6740576a6caea25571983d70f462376ad81fc51f5f34556978cda935668\": rpc error: code = NotFound desc = could not find container \"5d3bc6740576a6caea25571983d70f462376ad81fc51f5f34556978cda935668\": container with ID starting with 5d3bc6740576a6caea25571983d70f462376ad81fc51f5f34556978cda935668 not found: ID does not exist" Jul 23 12:17:31.268433 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:31.268401 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk" Jul 23 12:17:43.847508 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:43.847468 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk"] Jul 23 12:17:43.848028 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:43.847691 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk" podUID="06536fdd-1174-43e9-872a-624917a01d6c" containerName="manager" containerID="cri-o://a9bae3d9ed33d6fbb060805f211a394581ac8d14badb9a3e5b338ee2e331bfd2" gracePeriod=10 Jul 23 12:17:44.104940 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:44.104916 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk" Jul 23 12:17:44.117320 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:44.117288 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ml5wf\" (UniqueName: \"kubernetes.io/projected/06536fdd-1174-43e9-872a-624917a01d6c-kube-api-access-ml5wf\") pod \"06536fdd-1174-43e9-872a-624917a01d6c\" (UID: \"06536fdd-1174-43e9-872a-624917a01d6c\") " Jul 23 12:17:44.117510 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:44.117332 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/06536fdd-1174-43e9-872a-624917a01d6c-extensions-socket-volume\") pod \"06536fdd-1174-43e9-872a-624917a01d6c\" (UID: \"06536fdd-1174-43e9-872a-624917a01d6c\") " Jul 23 12:17:44.117862 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:44.117824 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06536fdd-1174-43e9-872a-624917a01d6c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "06536fdd-1174-43e9-872a-624917a01d6c" (UID: "06536fdd-1174-43e9-872a-624917a01d6c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:17:44.119367 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:44.119346 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06536fdd-1174-43e9-872a-624917a01d6c-kube-api-access-ml5wf" (OuterVolumeSpecName: "kube-api-access-ml5wf") pod "06536fdd-1174-43e9-872a-624917a01d6c" (UID: "06536fdd-1174-43e9-872a-624917a01d6c"). InnerVolumeSpecName "kube-api-access-ml5wf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:17:44.218822 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:44.218780 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ml5wf\" (UniqueName: \"kubernetes.io/projected/06536fdd-1174-43e9-872a-624917a01d6c-kube-api-access-ml5wf\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:17:44.218822 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:44.218817 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/06536fdd-1174-43e9-872a-624917a01d6c-extensions-socket-volume\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:17:44.372407 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:44.372313 2578 generic.go:356] "Generic (PLEG): container finished" podID="06536fdd-1174-43e9-872a-624917a01d6c" containerID="a9bae3d9ed33d6fbb060805f211a394581ac8d14badb9a3e5b338ee2e331bfd2" exitCode=0 Jul 23 12:17:44.372407 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:44.372358 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk" Jul 23 12:17:44.372620 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:44.372356 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk" event={"ID":"06536fdd-1174-43e9-872a-624917a01d6c","Type":"ContainerDied","Data":"a9bae3d9ed33d6fbb060805f211a394581ac8d14badb9a3e5b338ee2e331bfd2"} Jul 23 12:17:44.372620 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:44.372493 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk" event={"ID":"06536fdd-1174-43e9-872a-624917a01d6c","Type":"ContainerDied","Data":"83e2fbe2ee6ef886334c2e4ac9e5052311a3a5cf3040fef83b26ccf77e85da86"} Jul 23 12:17:44.372620 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:44.372524 2578 scope.go:117] "RemoveContainer" containerID="a9bae3d9ed33d6fbb060805f211a394581ac8d14badb9a3e5b338ee2e331bfd2" Jul 23 12:17:44.381790 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:44.381768 2578 scope.go:117] "RemoveContainer" containerID="a9bae3d9ed33d6fbb060805f211a394581ac8d14badb9a3e5b338ee2e331bfd2" Jul 23 12:17:44.382102 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:17:44.382078 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9bae3d9ed33d6fbb060805f211a394581ac8d14badb9a3e5b338ee2e331bfd2\": container with ID starting with a9bae3d9ed33d6fbb060805f211a394581ac8d14badb9a3e5b338ee2e331bfd2 not found: ID does not exist" containerID="a9bae3d9ed33d6fbb060805f211a394581ac8d14badb9a3e5b338ee2e331bfd2" Jul 23 12:17:44.382171 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:44.382111 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9bae3d9ed33d6fbb060805f211a394581ac8d14badb9a3e5b338ee2e331bfd2"} err="failed to get container status \"a9bae3d9ed33d6fbb060805f211a394581ac8d14badb9a3e5b338ee2e331bfd2\": rpc error: code = NotFound desc = could not find container \"a9bae3d9ed33d6fbb060805f211a394581ac8d14badb9a3e5b338ee2e331bfd2\": container with ID starting with a9bae3d9ed33d6fbb060805f211a394581ac8d14badb9a3e5b338ee2e331bfd2 not found: ID does not exist" Jul 23 12:17:44.397665 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:44.397626 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk"] Jul 23 12:17:44.402349 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:44.402322 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wzhvk"] Jul 23 12:17:44.935485 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:44.935454 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06536fdd-1174-43e9-872a-624917a01d6c" path="/var/lib/kubelet/pods/06536fdd-1174-43e9-872a-624917a01d6c/volumes" Jul 23 12:17:47.840876 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:47.840831 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4"] Jul 23 12:17:47.841378 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:47.841360 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f8f63e7-be2b-4242-ad70-791d25ee2448" containerName="manager" Jul 23 12:17:47.841378 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:47.841380 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f8f63e7-be2b-4242-ad70-791d25ee2448" containerName="manager" Jul 23 12:17:47.841453 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:47.841389 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06536fdd-1174-43e9-872a-624917a01d6c" containerName="manager" Jul 23 12:17:47.841453 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:47.841395 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="06536fdd-1174-43e9-872a-624917a01d6c" containerName="manager" Jul 23 12:17:47.841453 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:47.841450 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f8f63e7-be2b-4242-ad70-791d25ee2448" containerName="manager" Jul 23 12:17:47.841544 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:47.841461 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="06536fdd-1174-43e9-872a-624917a01d6c" containerName="manager" Jul 23 12:17:47.844485 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:47.844464 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:47.847141 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:47.847121 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-t77z5\"" Jul 23 12:17:47.857056 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:47.857030 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4"] Jul 23 12:17:47.948790 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:47.948747 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e5e2463-5b11-4730-83fa-29e4b172c94a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:47.949056 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:47.949030 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e5e2463-5b11-4730-83fa-29e4b172c94a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:47.949215 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:47.949072 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e5e2463-5b11-4730-83fa-29e4b172c94a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:47.949215 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:47.949101 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e5e2463-5b11-4730-83fa-29e4b172c94a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:47.949215 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:47.949162 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrq8d\" (UniqueName: \"kubernetes.io/projected/7e5e2463-5b11-4730-83fa-29e4b172c94a-kube-api-access-qrq8d\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:47.949389 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:47.949232 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e5e2463-5b11-4730-83fa-29e4b172c94a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:47.949389 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:47.949279 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e5e2463-5b11-4730-83fa-29e4b172c94a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:47.949389 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:47.949337 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e5e2463-5b11-4730-83fa-29e4b172c94a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:47.949389 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:47.949379 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e5e2463-5b11-4730-83fa-29e4b172c94a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:48.050377 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.050335 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e5e2463-5b11-4730-83fa-29e4b172c94a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:48.050377 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.050378 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e5e2463-5b11-4730-83fa-29e4b172c94a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:48.050598 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.050498 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e5e2463-5b11-4730-83fa-29e4b172c94a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:48.050598 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.050544 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e5e2463-5b11-4730-83fa-29e4b172c94a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:48.050598 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.050574 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e5e2463-5b11-4730-83fa-29e4b172c94a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:48.050743 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.050596 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e5e2463-5b11-4730-83fa-29e4b172c94a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:48.050743 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.050663 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrq8d\" (UniqueName: \"kubernetes.io/projected/7e5e2463-5b11-4730-83fa-29e4b172c94a-kube-api-access-qrq8d\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:48.050852 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.050749 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e5e2463-5b11-4730-83fa-29e4b172c94a-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:48.050852 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.050791 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e5e2463-5b11-4730-83fa-29e4b172c94a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:48.050852 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.050846 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e5e2463-5b11-4730-83fa-29e4b172c94a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:48.051025 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.050970 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e5e2463-5b11-4730-83fa-29e4b172c94a-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:48.051025 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.050990 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e5e2463-5b11-4730-83fa-29e4b172c94a-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:48.051171 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.051153 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e5e2463-5b11-4730-83fa-29e4b172c94a-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:48.051235 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.051179 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e5e2463-5b11-4730-83fa-29e4b172c94a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:48.053061 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.053030 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e5e2463-5b11-4730-83fa-29e4b172c94a-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:48.053324 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.053304 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e5e2463-5b11-4730-83fa-29e4b172c94a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:48.063667 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.063641 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e5e2463-5b11-4730-83fa-29e4b172c94a-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:48.063800 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.063781 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrq8d\" (UniqueName: \"kubernetes.io/projected/7e5e2463-5b11-4730-83fa-29e4b172c94a-kube-api-access-qrq8d\") pod \"maas-default-gateway-openshift-default-56654dcf7-mqbl4\" (UID: \"7e5e2463-5b11-4730-83fa-29e4b172c94a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:48.157127 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.157084 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:17:48.300312 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.300286 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4"] Jul 23 12:17:48.302753 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:17:48.302727 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e5e2463_5b11_4730_83fa_29e4b172c94a.slice/crio-0fbf2ebfc4bad895aba2c97f12f148eef73b1edf3c8db2b58ac859b40ce985d5 WatchSource:0}: Error finding container 0fbf2ebfc4bad895aba2c97f12f148eef73b1edf3c8db2b58ac859b40ce985d5: Status 404 returned error can't find the container with id 0fbf2ebfc4bad895aba2c97f12f148eef73b1edf3c8db2b58ac859b40ce985d5 Jul 23 12:17:48.393525 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:48.393491 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" event={"ID":"7e5e2463-5b11-4730-83fa-29e4b172c94a","Type":"ContainerStarted","Data":"0fbf2ebfc4bad895aba2c97f12f148eef73b1edf3c8db2b58ac859b40ce985d5"} Jul 23 12:17:59.357571 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:59.357527 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:17:59.358010 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:59.357602 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:17:59.358010 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:59.357636 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:17:59.447537 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:59.447500 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" event={"ID":"7e5e2463-5b11-4730-83fa-29e4b172c94a","Type":"ContainerStarted","Data":"3bbcd932533343e28ff44ffdfc433177f424f9a2cd1de234ba99101c8e3f3cf5"} Jul 23 12:17:59.474228 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:17:59.474151 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" podStartSLOduration=1.421567117 podStartE2EDuration="12.47413633s" totalImagesPullingTime="11.052569213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:17:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:17:59.472146001 +0000 UTC m=+491.064680696" watchObservedRunningTime="2026-07-23 12:17:59.47413633 +0000 UTC m=+491.066671022" Jul 23 12:18:00.157533 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:00.157485 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:18:00.162306 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:00.162278 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:18:00.453102 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:00.453014 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:18:00.453988 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:00.453970 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:18:02.125517 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:02.125486 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:18:02.128310 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:02.128288 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-wxtwj" Jul 23 12:18:02.131223 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:02.131201 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-224vt\"" Jul 23 12:18:02.131223 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:02.131212 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 23 12:18:02.131402 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:02.131385 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 23 12:18:02.132110 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:02.132095 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 23 12:18:02.141168 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:02.141146 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:18:02.163114 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:02.163083 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:18:02.189599 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:02.189561 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wr5k\" (UniqueName: \"kubernetes.io/projected/e6654821-3de4-4d0f-891d-7ad9cd0a936c-kube-api-access-2wr5k\") pod \"limitador-limitador-54d6d988d5-wxtwj\" (UID: \"e6654821-3de4-4d0f-891d-7ad9cd0a936c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wxtwj" Jul 23 12:18:02.189764 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:02.189620 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e6654821-3de4-4d0f-891d-7ad9cd0a936c-config-file\") pod \"limitador-limitador-54d6d988d5-wxtwj\" (UID: \"e6654821-3de4-4d0f-891d-7ad9cd0a936c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wxtwj" Jul 23 12:18:02.290667 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:02.290631 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wr5k\" (UniqueName: \"kubernetes.io/projected/e6654821-3de4-4d0f-891d-7ad9cd0a936c-kube-api-access-2wr5k\") pod \"limitador-limitador-54d6d988d5-wxtwj\" (UID: \"e6654821-3de4-4d0f-891d-7ad9cd0a936c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wxtwj" Jul 23 12:18:02.290838 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:02.290698 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e6654821-3de4-4d0f-891d-7ad9cd0a936c-config-file\") pod \"limitador-limitador-54d6d988d5-wxtwj\" (UID: \"e6654821-3de4-4d0f-891d-7ad9cd0a936c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wxtwj" Jul 23 12:18:02.291387 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:02.291367 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e6654821-3de4-4d0f-891d-7ad9cd0a936c-config-file\") pod \"limitador-limitador-54d6d988d5-wxtwj\" (UID: \"e6654821-3de4-4d0f-891d-7ad9cd0a936c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wxtwj" Jul 23 12:18:02.302257 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:02.302235 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wr5k\" (UniqueName: \"kubernetes.io/projected/e6654821-3de4-4d0f-891d-7ad9cd0a936c-kube-api-access-2wr5k\") pod \"limitador-limitador-54d6d988d5-wxtwj\" (UID: \"e6654821-3de4-4d0f-891d-7ad9cd0a936c\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wxtwj" Jul 23 12:18:02.438115 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:02.438027 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-wxtwj" Jul 23 12:18:02.576057 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:02.576033 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:18:02.578096 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:18:02.578068 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6654821_3de4_4d0f_891d_7ad9cd0a936c.slice/crio-2d01e303e92f6ff3bb126da817fe10ed8fe7383d3dd06faccf00abe6bf453add WatchSource:0}: Error finding container 2d01e303e92f6ff3bb126da817fe10ed8fe7383d3dd06faccf00abe6bf453add: Status 404 returned error can't find the container with id 2d01e303e92f6ff3bb126da817fe10ed8fe7383d3dd06faccf00abe6bf453add Jul 23 12:18:03.470396 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:03.470316 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-wxtwj" event={"ID":"e6654821-3de4-4d0f-891d-7ad9cd0a936c","Type":"ContainerStarted","Data":"2d01e303e92f6ff3bb126da817fe10ed8fe7383d3dd06faccf00abe6bf453add"} Jul 23 12:18:05.483250 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:05.483139 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-wxtwj" event={"ID":"e6654821-3de4-4d0f-891d-7ad9cd0a936c","Type":"ContainerStarted","Data":"c74a0d7b9050c53a19278eb41b25eb52a2f23ca9fca411bfb815bde71c15048b"} Jul 23 12:18:05.483656 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:05.483326 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-wxtwj" Jul 23 12:18:05.514738 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:05.514675 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-wxtwj" podStartSLOduration=0.905737136 podStartE2EDuration="3.514657509s" totalImagesPullingTime="2.608920373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:18:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:18:05.51203161 +0000 UTC m=+497.104566304" watchObservedRunningTime="2026-07-23 12:18:05.514657509 +0000 UTC m=+497.107192201" Jul 23 12:18:16.489698 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:18:16.489664 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-wxtwj" Jul 23 12:19:24.787804 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:24.787722 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-c5fqg"] Jul 23 12:19:24.790382 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:24.790365 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-c5fqg" Jul 23 12:19:24.792822 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:24.792799 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 23 12:19:24.792897 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:24.792880 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-hxjhx\"" Jul 23 12:19:24.800505 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:24.800480 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-c5fqg"] Jul 23 12:19:24.820432 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:24.820396 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:19:24.854156 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:24.854127 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a39e593-e5b5-4f09-bb0f-19bc95da799c-plugins-config-volume\") pod \"payload-processing-5bf47cd988-c5fqg\" (UID: \"3a39e593-e5b5-4f09-bb0f-19bc95da799c\") " pod="openshift-ingress/payload-processing-5bf47cd988-c5fqg" Jul 23 12:19:24.854331 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:24.854180 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljnb9\" (UniqueName: \"kubernetes.io/projected/3a39e593-e5b5-4f09-bb0f-19bc95da799c-kube-api-access-ljnb9\") pod \"payload-processing-5bf47cd988-c5fqg\" (UID: \"3a39e593-e5b5-4f09-bb0f-19bc95da799c\") " pod="openshift-ingress/payload-processing-5bf47cd988-c5fqg" Jul 23 12:19:24.955483 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:24.955450 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a39e593-e5b5-4f09-bb0f-19bc95da799c-plugins-config-volume\") pod \"payload-processing-5bf47cd988-c5fqg\" (UID: \"3a39e593-e5b5-4f09-bb0f-19bc95da799c\") " pod="openshift-ingress/payload-processing-5bf47cd988-c5fqg" Jul 23 12:19:24.955664 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:24.955498 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljnb9\" (UniqueName: \"kubernetes.io/projected/3a39e593-e5b5-4f09-bb0f-19bc95da799c-kube-api-access-ljnb9\") pod \"payload-processing-5bf47cd988-c5fqg\" (UID: \"3a39e593-e5b5-4f09-bb0f-19bc95da799c\") " pod="openshift-ingress/payload-processing-5bf47cd988-c5fqg" Jul 23 12:19:24.956312 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:24.956281 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a39e593-e5b5-4f09-bb0f-19bc95da799c-plugins-config-volume\") pod \"payload-processing-5bf47cd988-c5fqg\" (UID: \"3a39e593-e5b5-4f09-bb0f-19bc95da799c\") " pod="openshift-ingress/payload-processing-5bf47cd988-c5fqg" Jul 23 12:19:24.965160 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:24.965135 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljnb9\" (UniqueName: \"kubernetes.io/projected/3a39e593-e5b5-4f09-bb0f-19bc95da799c-kube-api-access-ljnb9\") pod \"payload-processing-5bf47cd988-c5fqg\" (UID: \"3a39e593-e5b5-4f09-bb0f-19bc95da799c\") " pod="openshift-ingress/payload-processing-5bf47cd988-c5fqg" Jul 23 12:19:25.102371 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:25.102340 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-c5fqg" Jul 23 12:19:25.234773 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:25.234737 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-c5fqg"] Jul 23 12:19:25.237734 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:19:25.237706 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a39e593_e5b5_4f09_bb0f_19bc95da799c.slice/crio-1e4bdb585dd7aa6ceda04c7e3e7bcd6c629e89ed54cddfe9a8260bd705541e1a WatchSource:0}: Error finding container 1e4bdb585dd7aa6ceda04c7e3e7bcd6c629e89ed54cddfe9a8260bd705541e1a: Status 404 returned error can't find the container with id 1e4bdb585dd7aa6ceda04c7e3e7bcd6c629e89ed54cddfe9a8260bd705541e1a Jul 23 12:19:25.866518 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:25.866475 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-c5fqg" event={"ID":"3a39e593-e5b5-4f09-bb0f-19bc95da799c","Type":"ContainerStarted","Data":"1e4bdb585dd7aa6ceda04c7e3e7bcd6c629e89ed54cddfe9a8260bd705541e1a"} Jul 23 12:19:28.887529 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:28.887489 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-c5fqg" event={"ID":"3a39e593-e5b5-4f09-bb0f-19bc95da799c","Type":"ContainerStarted","Data":"3a0a8d8d17b71278a4911253bc0fb4d17ae5fdfc4bd732da0cca2d315c28d59d"} Jul 23 12:19:28.887935 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:28.887609 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-c5fqg" Jul 23 12:19:28.910517 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:28.907835 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-c5fqg" podStartSLOduration=2.107399963 podStartE2EDuration="4.907814747s" totalImagesPullingTime="2.800414784s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:19:28.906528893 +0000 UTC m=+580.499063589" watchObservedRunningTime="2026-07-23 12:19:28.907814747 +0000 UTC m=+580.500349441" Jul 23 12:19:31.494397 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:31.494351 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mqbl4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 23 12:19:31.494887 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:31.494424 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" podUID="7e5e2463-5b11-4730-83fa-29e4b172c94a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 23 12:19:32.910482 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:32.910447 2578 generic.go:356] "Generic (PLEG): container finished" podID="7e5e2463-5b11-4730-83fa-29e4b172c94a" containerID="3bbcd932533343e28ff44ffdfc433177f424f9a2cd1de234ba99101c8e3f3cf5" exitCode=0 Jul 23 12:19:32.910894 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:32.910525 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" event={"ID":"7e5e2463-5b11-4730-83fa-29e4b172c94a","Type":"ContainerDied","Data":"3bbcd932533343e28ff44ffdfc433177f424f9a2cd1de234ba99101c8e3f3cf5"} Jul 23 12:19:32.911021 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:32.911005 2578 scope.go:117] "RemoveContainer" containerID="3bbcd932533343e28ff44ffdfc433177f424f9a2cd1de234ba99101c8e3f3cf5" Jul 23 12:19:32.912085 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:32.912031 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:19:32.912138 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:32.912100 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:19:32.912138 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:32.912131 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:19:33.920808 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:33.920750 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" event={"ID":"7e5e2463-5b11-4730-83fa-29e4b172c94a","Type":"ContainerStarted","Data":"2ee020fc1769bc23b4266a33dedf3874dfd714503e441cedd15b73a5159bcac0"} Jul 23 12:19:34.157305 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:34.157266 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:19:34.157603 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:34.157580 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mqbl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 23 12:19:34.157652 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:34.157629 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" podUID="7e5e2463-5b11-4730-83fa-29e4b172c94a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 23 12:19:35.158129 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:35.158088 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mqbl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 23 12:19:35.158547 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:35.158153 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" podUID="7e5e2463-5b11-4730-83fa-29e4b172c94a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 23 12:19:36.158221 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:36.158159 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mqbl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 23 12:19:36.158608 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:36.158247 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" podUID="7e5e2463-5b11-4730-83fa-29e4b172c94a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 23 12:19:37.157902 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:37.157868 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mqbl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 23 12:19:37.158106 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:37.157929 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" podUID="7e5e2463-5b11-4730-83fa-29e4b172c94a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 23 12:19:38.161590 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:38.161563 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:19:38.161989 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:38.161816 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:19:38.162580 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:38.162560 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mqbl4" Jul 23 12:19:39.894161 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:39.894126 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-c5fqg" Jul 23 12:19:41.210435 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:41.210399 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:19:48.855158 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:48.855129 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:19:48.856221 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:19:48.856181 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:20:09.895095 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:09.895056 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:20:18.595543 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.595459 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5"] Jul 23 12:20:18.602575 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.599617 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:18.605325 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.605297 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 23 12:20:18.605628 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.605320 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 23 12:20:18.605995 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.605980 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4n8m9\"" Jul 23 12:20:18.606897 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.606875 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 23 12:20:18.609911 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.609886 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5"] Jul 23 12:20:18.755717 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.755678 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2904f0a7-2a7c-4cef-aef8-dc8f20b12193-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwf5\" (UID: \"2904f0a7-2a7c-4cef-aef8-dc8f20b12193\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:18.755906 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.755755 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2904f0a7-2a7c-4cef-aef8-dc8f20b12193-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwf5\" (UID: \"2904f0a7-2a7c-4cef-aef8-dc8f20b12193\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:18.755906 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.755785 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2904f0a7-2a7c-4cef-aef8-dc8f20b12193-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwf5\" (UID: \"2904f0a7-2a7c-4cef-aef8-dc8f20b12193\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:18.755906 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.755822 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvr9n\" (UniqueName: \"kubernetes.io/projected/2904f0a7-2a7c-4cef-aef8-dc8f20b12193-kube-api-access-gvr9n\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwf5\" (UID: \"2904f0a7-2a7c-4cef-aef8-dc8f20b12193\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:18.755906 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.755852 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2904f0a7-2a7c-4cef-aef8-dc8f20b12193-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwf5\" (UID: \"2904f0a7-2a7c-4cef-aef8-dc8f20b12193\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:18.755906 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.755890 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2904f0a7-2a7c-4cef-aef8-dc8f20b12193-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwf5\" (UID: \"2904f0a7-2a7c-4cef-aef8-dc8f20b12193\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:18.856983 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.856946 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2904f0a7-2a7c-4cef-aef8-dc8f20b12193-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwf5\" (UID: \"2904f0a7-2a7c-4cef-aef8-dc8f20b12193\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:18.857233 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.857012 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2904f0a7-2a7c-4cef-aef8-dc8f20b12193-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwf5\" (UID: \"2904f0a7-2a7c-4cef-aef8-dc8f20b12193\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:18.857233 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.857041 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2904f0a7-2a7c-4cef-aef8-dc8f20b12193-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwf5\" (UID: \"2904f0a7-2a7c-4cef-aef8-dc8f20b12193\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:18.857233 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.857075 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvr9n\" (UniqueName: \"kubernetes.io/projected/2904f0a7-2a7c-4cef-aef8-dc8f20b12193-kube-api-access-gvr9n\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwf5\" (UID: \"2904f0a7-2a7c-4cef-aef8-dc8f20b12193\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:18.857233 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.857102 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2904f0a7-2a7c-4cef-aef8-dc8f20b12193-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwf5\" (UID: \"2904f0a7-2a7c-4cef-aef8-dc8f20b12193\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:18.857233 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.857142 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2904f0a7-2a7c-4cef-aef8-dc8f20b12193-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwf5\" (UID: \"2904f0a7-2a7c-4cef-aef8-dc8f20b12193\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:18.857563 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.857447 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2904f0a7-2a7c-4cef-aef8-dc8f20b12193-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwf5\" (UID: \"2904f0a7-2a7c-4cef-aef8-dc8f20b12193\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:18.857563 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.857470 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2904f0a7-2a7c-4cef-aef8-dc8f20b12193-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwf5\" (UID: \"2904f0a7-2a7c-4cef-aef8-dc8f20b12193\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:18.857563 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.857511 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2904f0a7-2a7c-4cef-aef8-dc8f20b12193-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwf5\" (UID: \"2904f0a7-2a7c-4cef-aef8-dc8f20b12193\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:18.859348 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.859328 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2904f0a7-2a7c-4cef-aef8-dc8f20b12193-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwf5\" (UID: \"2904f0a7-2a7c-4cef-aef8-dc8f20b12193\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:18.859498 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.859479 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2904f0a7-2a7c-4cef-aef8-dc8f20b12193-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwf5\" (UID: \"2904f0a7-2a7c-4cef-aef8-dc8f20b12193\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:18.867276 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.867242 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvr9n\" (UniqueName: \"kubernetes.io/projected/2904f0a7-2a7c-4cef-aef8-dc8f20b12193-kube-api-access-gvr9n\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hzwf5\" (UID: \"2904f0a7-2a7c-4cef-aef8-dc8f20b12193\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:18.915306 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:18.915271 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:19.049546 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:19.049513 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5"] Jul 23 12:20:19.053422 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:20:19.053390 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2904f0a7_2a7c_4cef_aef8_dc8f20b12193.slice/crio-2698582baed5a810a9dfb8b91c8dacea14e9572adda9f606e06dbc384c5c7107 WatchSource:0}: Error finding container 2698582baed5a810a9dfb8b91c8dacea14e9572adda9f606e06dbc384c5c7107: Status 404 returned error can't find the container with id 2698582baed5a810a9dfb8b91c8dacea14e9572adda9f606e06dbc384c5c7107 Jul 23 12:20:19.055031 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:19.055012 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 12:20:19.143667 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:19.143576 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" event={"ID":"2904f0a7-2a7c-4cef-aef8-dc8f20b12193","Type":"ContainerStarted","Data":"2698582baed5a810a9dfb8b91c8dacea14e9572adda9f606e06dbc384c5c7107"} Jul 23 12:20:19.911361 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:19.911326 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:20:21.154677 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:21.154637 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" event={"ID":"2904f0a7-2a7c-4cef-aef8-dc8f20b12193","Type":"ContainerStarted","Data":"de96e1536cb53cc4ac3b8dd27fb9e30a4ef6737df39aa7ed8079f4574dbf18d1"} Jul 23 12:20:21.178351 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:21.178298 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" podStartSLOduration=2.153424794 podStartE2EDuration="3.178280611s" totalImagesPullingTime="1.024855817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:20:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:20:21.176861675 +0000 UTC m=+632.769396367" watchObservedRunningTime="2026-07-23 12:20:21.178280611 +0000 UTC m=+632.770815305" Jul 23 12:20:28.915406 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:28.915368 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:28.915897 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:28.915438 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:28.927960 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:28.927936 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:29.210260 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:29.210158 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hzwf5" Jul 23 12:20:39.096966 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:39.096924 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:20:47.890511 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:20:47.890467 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:21:03.807465 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:21:03.807430 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:22:59.000411 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:22:59.000374 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:23:08.300901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:23:08.300859 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:23:20.794764 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:23:20.794681 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:24:48.887634 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:24:48.887563 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:24:48.887634 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:24:48.887577 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:25:01.102916 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:25:01.102881 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:25:41.199028 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:25:41.198992 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:25:50.906847 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:25:50.906808 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:25:59.199462 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:25:59.199424 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:26:09.002370 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:26:09.002330 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:26:19.112155 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:26:19.112067 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:26:28.712656 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:26:28.712615 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:26:41.813791 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:26:41.813714 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:26:50.822787 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:26:50.822756 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:27:13.119154 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:27:13.119117 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:27:25.199412 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:27:25.199378 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:27:49.005126 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:27:49.005048 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:27:59.600456 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:27:59.600415 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:28:14.706572 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:28:14.706534 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:28:26.007493 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:28:26.007453 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:28:36.216717 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:28:36.216676 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:28:45.512482 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:28:45.512447 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:28:55.108475 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:28:55.108439 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:29:48.918612 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:29:48.918581 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:29:48.921183 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:29:48.919893 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:30:42.802301 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:30:42.802208 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:30:54.393892 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:30:54.393856 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:31:02.998063 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:31:02.998030 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:31:15.207878 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:31:15.207844 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:31:25.998722 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:31:25.998683 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:31:36.521978 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:31:36.521937 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:31:46.201618 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:31:46.201579 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:31:57.011520 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:31:57.011486 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:32:06.596257 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:32:06.596207 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:32:17.308449 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:32:17.308371 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:32:28.199423 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:32:28.199382 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:32:49.407452 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:32:49.407406 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:32:59.094502 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:32:59.094464 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:33:12.303944 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:33:12.303910 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:33:27.498004 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:33:27.497967 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:33:56.308812 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:33:56.308724 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:34:12.009822 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:34:12.009788 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:34:19.010961 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:34:19.010929 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:34:35.415018 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:34:35.414979 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:34:45.327742 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:34:45.327709 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:34:48.952109 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:34:48.951997 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:34:48.955516 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:34:48.955014 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:34:55.198572 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:34:55.198533 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:35:05.115688 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:35:05.115649 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:35:14.511315 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:35:14.511215 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:35:23.816958 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:35:23.816918 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:35:36.909004 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:35:36.908961 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:35:45.901906 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:35:45.901874 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:35:55.407932 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:35:55.407900 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:36:04.798181 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:36:04.798143 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:36:17.800683 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:36:17.800647 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:36:27.305291 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:36:27.305254 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:36:39.293400 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:36:39.293359 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:36:49.301429 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:36:49.301347 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:37:03.402413 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:37:03.402374 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:37:12.198835 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:37:12.198800 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:37:35.000563 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:37:35.000528 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:37:50.104693 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:37:50.104658 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:38:08.428482 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:08.428438 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-543ade38-sbk65"] Jul 23 12:38:08.432028 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:08.432009 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-543ade38-sbk65" Jul 23 12:38:08.434790 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:08.434759 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ncgg\"" Jul 23 12:38:08.434918 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:08.434786 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 12:38:08.440230 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:08.440206 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-543ade38-sbk65"] Jul 23 12:38:08.474664 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:08.474624 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9f608cc-8b0f-4e3b-b292-271bbe9875f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-543ade38-sbk65\" (UID: \"e9f608cc-8b0f-4e3b-b292-271bbe9875f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-543ade38-sbk65" Jul 23 12:38:08.576163 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:08.576125 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9f608cc-8b0f-4e3b-b292-271bbe9875f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-543ade38-sbk65\" (UID: \"e9f608cc-8b0f-4e3b-b292-271bbe9875f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-543ade38-sbk65" Jul 23 12:38:08.576766 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:08.576745 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9f608cc-8b0f-4e3b-b292-271bbe9875f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-543ade38-sbk65\" (UID: \"e9f608cc-8b0f-4e3b-b292-271bbe9875f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-543ade38-sbk65" Jul 23 12:38:08.743501 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:08.743407 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-543ade38-sbk65" Jul 23 12:38:08.877717 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:08.877691 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-543ade38-sbk65"] Jul 23 12:38:08.880248 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:38:08.880219 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9f608cc_8b0f_4e3b_b292_271bbe9875f1.slice/crio-88b9895bb7136a83603e4de3d3ef610e034bdef0c6164981289e2801d5db8128 WatchSource:0}: Error finding container 88b9895bb7136a83603e4de3d3ef610e034bdef0c6164981289e2801d5db8128: Status 404 returned error can't find the container with id 88b9895bb7136a83603e4de3d3ef610e034bdef0c6164981289e2801d5db8128 Jul 23 12:38:08.882169 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:08.882152 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 12:38:09.390576 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:09.390537 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-543ade38-sbk65" event={"ID":"e9f608cc-8b0f-4e3b-b292-271bbe9875f1","Type":"ContainerStarted","Data":"88b9895bb7136a83603e4de3d3ef610e034bdef0c6164981289e2801d5db8128"} Jul 23 12:38:14.421826 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:14.421786 2578 generic.go:356] "Generic (PLEG): container finished" podID="e9f608cc-8b0f-4e3b-b292-271bbe9875f1" containerID="68b88aebe61ea04820197894ce4a4ac4e464ef7cbcc0a028d3ee9a8debee1c4e" exitCode=0 Jul 23 12:38:14.422222 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:14.421868 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-543ade38-sbk65" event={"ID":"e9f608cc-8b0f-4e3b-b292-271bbe9875f1","Type":"ContainerDied","Data":"68b88aebe61ea04820197894ce4a4ac4e464ef7cbcc0a028d3ee9a8debee1c4e"} Jul 23 12:38:15.562341 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:15.562317 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-543ade38-sbk65" Jul 23 12:38:15.644621 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:15.644585 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9f608cc-8b0f-4e3b-b292-271bbe9875f1-maas-api-service-ca\") pod \"e9f608cc-8b0f-4e3b-b292-271bbe9875f1\" (UID: \"e9f608cc-8b0f-4e3b-b292-271bbe9875f1\") " Jul 23 12:38:15.644969 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:15.644946 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9f608cc-8b0f-4e3b-b292-271bbe9875f1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e9f608cc-8b0f-4e3b-b292-271bbe9875f1" (UID: "e9f608cc-8b0f-4e3b-b292-271bbe9875f1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:38:15.745572 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:15.745492 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9f608cc-8b0f-4e3b-b292-271bbe9875f1-maas-api-service-ca\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:38:16.433365 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:16.433334 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-543ade38-sbk65" Jul 23 12:38:16.433538 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:16.433372 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-543ade38-sbk65" event={"ID":"e9f608cc-8b0f-4e3b-b292-271bbe9875f1","Type":"ContainerDied","Data":"88b9895bb7136a83603e4de3d3ef610e034bdef0c6164981289e2801d5db8128"} Jul 23 12:38:16.433538 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:16.433412 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="88b9895bb7136a83603e4de3d3ef610e034bdef0c6164981289e2801d5db8128" Jul 23 12:38:21.711210 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:21.711159 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:38:23.517890 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:23.517853 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-543ade38-sbk65"] Jul 23 12:38:23.521047 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:23.521021 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-543ade38-sbk65"] Jul 23 12:38:24.748817 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.748772 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w"] Jul 23 12:38:24.749403 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.749374 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9f608cc-8b0f-4e3b-b292-271bbe9875f1" containerName="revoke-keys" Jul 23 12:38:24.749403 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.749398 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9f608cc-8b0f-4e3b-b292-271bbe9875f1" containerName="revoke-keys" Jul 23 12:38:24.749675 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.749515 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9f608cc-8b0f-4e3b-b292-271bbe9875f1" containerName="revoke-keys" Jul 23 12:38:24.752828 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.752809 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.755987 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.755966 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-2cd5664e-gw-openshift-default-dockercfg-9wqj6\"" Jul 23 12:38:24.764959 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.764926 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w"] Jul 23 12:38:24.839852 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.839820 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-data\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.839852 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.839861 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-workload-certs\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.840060 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.839883 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-envoy\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.840060 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.839909 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-token\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.840060 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.840021 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-workload-socket\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.840060 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.840051 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-credential-socket\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.840222 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.840075 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istiod-ca-cert\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.840222 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.840175 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpv4x\" (UniqueName: \"kubernetes.io/projected/8b026a5c-c651-4634-b1ce-fb3e37b84d16-kube-api-access-rpv4x\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.840296 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.840226 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-podinfo\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.935254 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.935222 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9f608cc-8b0f-4e3b-b292-271bbe9875f1" path="/var/lib/kubelet/pods/e9f608cc-8b0f-4e3b-b292-271bbe9875f1/volumes" Jul 23 12:38:24.941548 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.941522 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-envoy\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.941785 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.941557 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-token\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.941785 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.941651 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-workload-socket\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.941785 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.941680 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-credential-socket\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.941785 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.941712 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istiod-ca-cert\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.941785 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.941767 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpv4x\" (UniqueName: \"kubernetes.io/projected/8b026a5c-c651-4634-b1ce-fb3e37b84d16-kube-api-access-rpv4x\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.942067 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.941800 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-podinfo\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.942067 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.941832 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-data\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.942067 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.941867 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-workload-certs\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.942270 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.942063 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-workload-socket\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.942270 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.942093 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-credential-socket\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.942407 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.942385 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-workload-certs\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.942470 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.942448 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-data\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.942917 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.942892 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istiod-ca-cert\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.949805 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.949779 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-podinfo\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.950774 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.950748 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-envoy\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.951901 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.951866 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpv4x\" (UniqueName: \"kubernetes.io/projected/8b026a5c-c651-4634-b1ce-fb3e37b84d16-kube-api-access-rpv4x\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:24.952742 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:24.952719 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-token\") pod \"e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:25.063958 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:25.063858 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:25.202704 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:25.202668 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w"] Jul 23 12:38:25.207351 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:38:25.207319 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b026a5c_c651_4634_b1ce_fb3e37b84d16.slice/crio-a33755bb38fc2b40427f576e5f32660b87906a3ac19d141403bf0877e23e8bd7 WatchSource:0}: Error finding container a33755bb38fc2b40427f576e5f32660b87906a3ac19d141403bf0877e23e8bd7: Status 404 returned error can't find the container with id a33755bb38fc2b40427f576e5f32660b87906a3ac19d141403bf0877e23e8bd7 Jul 23 12:38:25.209785 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:25.209744 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:38:25.209860 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:25.209833 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:38:25.209899 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:25.209878 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:38:25.477973 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:25.477929 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" event={"ID":"8b026a5c-c651-4634-b1ce-fb3e37b84d16","Type":"ContainerStarted","Data":"01e3be70308bf3a24cea4b1fa5d72d7c9142e5702586f57b088cb5814e312553"} Jul 23 12:38:25.478162 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:25.477996 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" event={"ID":"8b026a5c-c651-4634-b1ce-fb3e37b84d16","Type":"ContainerStarted","Data":"a33755bb38fc2b40427f576e5f32660b87906a3ac19d141403bf0877e23e8bd7"} Jul 23 12:38:25.499830 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:25.499783 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" podStartSLOduration=1.499768563 podStartE2EDuration="1.499768563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:38:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:38:25.498799331 +0000 UTC m=+1717.091334035" watchObservedRunningTime="2026-07-23 12:38:25.499768563 +0000 UTC m=+1717.092303340" Jul 23 12:38:26.065007 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:26.064975 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:26.069607 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:26.069582 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:26.483238 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:26.483186 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:26.484227 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:26.484207 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:32.901718 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:32.901675 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:38:41.485302 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:41.485254 2578 patch_prober.go:28] interesting pod/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 23 12:38:41.485801 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:41.485334 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" podUID="8b026a5c-c651-4634-b1ce-fb3e37b84d16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 23 12:38:41.564815 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:41.564783 2578 generic.go:356] "Generic (PLEG): container finished" podID="8b026a5c-c651-4634-b1ce-fb3e37b84d16" containerID="01e3be70308bf3a24cea4b1fa5d72d7c9142e5702586f57b088cb5814e312553" exitCode=0 Jul 23 12:38:41.564996 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:41.564822 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" event={"ID":"8b026a5c-c651-4634-b1ce-fb3e37b84d16","Type":"ContainerDied","Data":"01e3be70308bf3a24cea4b1fa5d72d7c9142e5702586f57b088cb5814e312553"} Jul 23 12:38:41.565254 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:41.565241 2578 scope.go:117] "RemoveContainer" containerID="01e3be70308bf3a24cea4b1fa5d72d7c9142e5702586f57b088cb5814e312553" Jul 23 12:38:41.566282 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:41.566256 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:38:41.566346 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:41.566324 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:38:41.566395 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:41.566354 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:38:42.571642 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:42.571604 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" event={"ID":"8b026a5c-c651-4634-b1ce-fb3e37b84d16","Type":"ContainerStarted","Data":"d85ae8a5023f87b7decc4333738ec0bd75db037c24e34f6cb320f2ab21e6211f"} Jul 23 12:38:43.064384 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:43.064345 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:43.064679 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:43.064653 2578 patch_prober.go:28] interesting pod/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 23 12:38:43.064782 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:43.064730 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" podUID="8b026a5c-c651-4634-b1ce-fb3e37b84d16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 23 12:38:44.064910 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:44.064868 2578 patch_prober.go:28] interesting pod/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 23 12:38:44.065344 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:44.064941 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" podUID="8b026a5c-c651-4634-b1ce-fb3e37b84d16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 23 12:38:45.064590 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:45.064552 2578 patch_prober.go:28] interesting pod/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 23 12:38:45.064776 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:45.064629 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" podUID="8b026a5c-c651-4634-b1ce-fb3e37b84d16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 23 12:38:46.064357 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:46.064317 2578 patch_prober.go:28] interesting pod/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 23 12:38:46.064777 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:46.064375 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" podUID="8b026a5c-c651-4634-b1ce-fb3e37b84d16" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 23 12:38:47.069073 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:47.069037 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:47.069496 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:47.069415 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:38:47.070106 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:38:47.070085 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:39:05.592030 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:05.591990 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:39:07.254093 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:07.254058 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w"] Jul 23 12:39:07.254539 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:07.254355 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" podUID="8b026a5c-c651-4634-b1ce-fb3e37b84d16" containerName="istio-proxy" containerID="cri-o://d85ae8a5023f87b7decc4333738ec0bd75db037c24e34f6cb320f2ab21e6211f" gracePeriod=30 Jul 23 12:39:12.506956 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.506921 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:39:12.580576 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.580498 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-credential-socket\") pod \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " Jul 23 12:39:12.580729 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.580581 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-token\") pod \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " Jul 23 12:39:12.580729 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.580615 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-envoy\") pod \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " Jul 23 12:39:12.580729 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.580644 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-podinfo\") pod \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " Jul 23 12:39:12.580935 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.580761 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8b026a5c-c651-4634-b1ce-fb3e37b84d16" (UID: "8b026a5c-c651-4634-b1ce-fb3e37b84d16"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:39:12.580935 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.580774 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpv4x\" (UniqueName: \"kubernetes.io/projected/8b026a5c-c651-4634-b1ce-fb3e37b84d16-kube-api-access-rpv4x\") pod \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " Jul 23 12:39:12.580935 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.580873 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-data\") pod \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " Jul 23 12:39:12.580935 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.580909 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-workload-certs\") pod \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " Jul 23 12:39:12.581150 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.580942 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-workload-socket\") pod \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " Jul 23 12:39:12.581150 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.580990 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istiod-ca-cert\") pod \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\" (UID: \"8b026a5c-c651-4634-b1ce-fb3e37b84d16\") " Jul 23 12:39:12.581150 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.581133 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8b026a5c-c651-4634-b1ce-fb3e37b84d16" (UID: "8b026a5c-c651-4634-b1ce-fb3e37b84d16"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:39:12.581356 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.581302 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8b026a5c-c651-4634-b1ce-fb3e37b84d16" (UID: "8b026a5c-c651-4634-b1ce-fb3e37b84d16"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:39:12.581446 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.581425 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-credential-socket\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:39:12.581508 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.581452 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-workload-certs\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:39:12.581508 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.581468 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-workload-socket\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:39:12.581625 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.581558 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8b026a5c-c651-4634-b1ce-fb3e37b84d16" (UID: "8b026a5c-c651-4634-b1ce-fb3e37b84d16"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:39:12.581836 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.581804 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-data" (OuterVolumeSpecName: "istio-data") pod "8b026a5c-c651-4634-b1ce-fb3e37b84d16" (UID: "8b026a5c-c651-4634-b1ce-fb3e37b84d16"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:39:12.582940 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.582909 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8b026a5c-c651-4634-b1ce-fb3e37b84d16" (UID: "8b026a5c-c651-4634-b1ce-fb3e37b84d16"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:39:12.582940 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.582921 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-token" (OuterVolumeSpecName: "istio-token") pod "8b026a5c-c651-4634-b1ce-fb3e37b84d16" (UID: "8b026a5c-c651-4634-b1ce-fb3e37b84d16"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:39:12.583077 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.582996 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b026a5c-c651-4634-b1ce-fb3e37b84d16-kube-api-access-rpv4x" (OuterVolumeSpecName: "kube-api-access-rpv4x") pod "8b026a5c-c651-4634-b1ce-fb3e37b84d16" (UID: "8b026a5c-c651-4634-b1ce-fb3e37b84d16"). InnerVolumeSpecName "kube-api-access-rpv4x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:39:12.583077 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.583012 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8b026a5c-c651-4634-b1ce-fb3e37b84d16" (UID: "8b026a5c-c651-4634-b1ce-fb3e37b84d16"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 12:39:12.682680 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.682645 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-data\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:39:12.682680 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.682676 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istiod-ca-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:39:12.682680 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.682685 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-token\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:39:12.682911 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.682693 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-envoy\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:39:12.682911 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.682702 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b026a5c-c651-4634-b1ce-fb3e37b84d16-istio-podinfo\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:39:12.682911 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.682710 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpv4x\" (UniqueName: \"kubernetes.io/projected/8b026a5c-c651-4634-b1ce-fb3e37b84d16-kube-api-access-rpv4x\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:39:12.733398 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.733361 2578 generic.go:356] "Generic (PLEG): container finished" podID="8b026a5c-c651-4634-b1ce-fb3e37b84d16" containerID="d85ae8a5023f87b7decc4333738ec0bd75db037c24e34f6cb320f2ab21e6211f" exitCode=0 Jul 23 12:39:12.733574 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.733406 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" Jul 23 12:39:12.733574 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.733444 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" event={"ID":"8b026a5c-c651-4634-b1ce-fb3e37b84d16","Type":"ContainerDied","Data":"d85ae8a5023f87b7decc4333738ec0bd75db037c24e34f6cb320f2ab21e6211f"} Jul 23 12:39:12.733574 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.733509 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w" event={"ID":"8b026a5c-c651-4634-b1ce-fb3e37b84d16","Type":"ContainerDied","Data":"a33755bb38fc2b40427f576e5f32660b87906a3ac19d141403bf0877e23e8bd7"} Jul 23 12:39:12.733574 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.733542 2578 scope.go:117] "RemoveContainer" containerID="d85ae8a5023f87b7decc4333738ec0bd75db037c24e34f6cb320f2ab21e6211f" Jul 23 12:39:12.743087 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.743069 2578 scope.go:117] "RemoveContainer" containerID="01e3be70308bf3a24cea4b1fa5d72d7c9142e5702586f57b088cb5814e312553" Jul 23 12:39:12.757724 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.757703 2578 scope.go:117] "RemoveContainer" containerID="d85ae8a5023f87b7decc4333738ec0bd75db037c24e34f6cb320f2ab21e6211f" Jul 23 12:39:12.758018 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:39:12.757998 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d85ae8a5023f87b7decc4333738ec0bd75db037c24e34f6cb320f2ab21e6211f\": container with ID starting with d85ae8a5023f87b7decc4333738ec0bd75db037c24e34f6cb320f2ab21e6211f not found: ID does not exist" containerID="d85ae8a5023f87b7decc4333738ec0bd75db037c24e34f6cb320f2ab21e6211f" Jul 23 12:39:12.758082 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.758026 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d85ae8a5023f87b7decc4333738ec0bd75db037c24e34f6cb320f2ab21e6211f"} err="failed to get container status \"d85ae8a5023f87b7decc4333738ec0bd75db037c24e34f6cb320f2ab21e6211f\": rpc error: code = NotFound desc = could not find container \"d85ae8a5023f87b7decc4333738ec0bd75db037c24e34f6cb320f2ab21e6211f\": container with ID starting with d85ae8a5023f87b7decc4333738ec0bd75db037c24e34f6cb320f2ab21e6211f not found: ID does not exist" Jul 23 12:39:12.758082 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.758042 2578 scope.go:117] "RemoveContainer" containerID="01e3be70308bf3a24cea4b1fa5d72d7c9142e5702586f57b088cb5814e312553" Jul 23 12:39:12.758282 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:39:12.758266 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01e3be70308bf3a24cea4b1fa5d72d7c9142e5702586f57b088cb5814e312553\": container with ID starting with 01e3be70308bf3a24cea4b1fa5d72d7c9142e5702586f57b088cb5814e312553 not found: ID does not exist" containerID="01e3be70308bf3a24cea4b1fa5d72d7c9142e5702586f57b088cb5814e312553" Jul 23 12:39:12.758355 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.758286 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01e3be70308bf3a24cea4b1fa5d72d7c9142e5702586f57b088cb5814e312553"} err="failed to get container status \"01e3be70308bf3a24cea4b1fa5d72d7c9142e5702586f57b088cb5814e312553\": rpc error: code = NotFound desc = could not find container \"01e3be70308bf3a24cea4b1fa5d72d7c9142e5702586f57b088cb5814e312553\": container with ID starting with 01e3be70308bf3a24cea4b1fa5d72d7c9142e5702586f57b088cb5814e312553 not found: ID does not exist" Jul 23 12:39:12.758729 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.758707 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w"] Jul 23 12:39:12.767105 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.767077 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-2cd5664e-gw-openshift-default-f498b95ff-lvf2w"] Jul 23 12:39:12.934749 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:12.934716 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b026a5c-c651-4634-b1ce-fb3e37b84d16" path="/var/lib/kubelet/pods/8b026a5c-c651-4634-b1ce-fb3e37b84d16/volumes" Jul 23 12:39:17.917984 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:17.917945 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:39:48.980577 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:48.980547 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:39:48.985530 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:48.985508 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:39:49.801469 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:49.801435 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:39:53.383394 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.383362 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg"] Jul 23 12:39:53.383794 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.383728 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b026a5c-c651-4634-b1ce-fb3e37b84d16" containerName="istio-proxy" Jul 23 12:39:53.383794 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.383740 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b026a5c-c651-4634-b1ce-fb3e37b84d16" containerName="istio-proxy" Jul 23 12:39:53.383794 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.383753 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b026a5c-c651-4634-b1ce-fb3e37b84d16" containerName="istio-proxy" Jul 23 12:39:53.383794 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.383759 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b026a5c-c651-4634-b1ce-fb3e37b84d16" containerName="istio-proxy" Jul 23 12:39:53.384023 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.383829 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b026a5c-c651-4634-b1ce-fb3e37b84d16" containerName="istio-proxy" Jul 23 12:39:53.386875 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.386854 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.389470 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.389450 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-3e72e548-openshift-default-dockercfg-n2sf2\"" Jul 23 12:39:53.397239 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.397216 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg"] Jul 23 12:39:53.471711 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.471679 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-data\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.471711 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.471714 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4l9m\" (UniqueName: \"kubernetes.io/projected/54eb1f5d-b743-41f0-bd9c-aac481e400f7-kube-api-access-f4l9m\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.471969 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.471744 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-workload-certs\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.471969 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.471863 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-credential-socket\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.471969 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.471910 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istiod-ca-cert\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.472130 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.472003 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-podinfo\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.472130 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.472048 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-envoy\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.472130 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.472122 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-workload-socket\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.472305 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.472206 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-token\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.573047 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.572996 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-credential-socket\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.573286 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.573062 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istiod-ca-cert\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.573286 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.573099 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-podinfo\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.573286 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.573133 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-envoy\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.573286 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.573188 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-workload-socket\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.573286 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.573277 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-token\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.573571 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.573330 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-data\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.573571 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.573354 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4l9m\" (UniqueName: \"kubernetes.io/projected/54eb1f5d-b743-41f0-bd9c-aac481e400f7-kube-api-access-f4l9m\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.573571 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.573383 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-workload-certs\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.573728 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.573590 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-workload-socket\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.573728 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.573673 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-data\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.573834 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.573758 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-workload-certs\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.573980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.573955 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-credential-socket\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.574103 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.573991 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istiod-ca-cert\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.575482 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.575457 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-envoy\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.575672 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.575655 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-podinfo\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.583872 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.583850 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-token\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.584126 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.584100 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4l9m\" (UniqueName: \"kubernetes.io/projected/54eb1f5d-b743-41f0-bd9c-aac481e400f7-kube-api-access-f4l9m\") pod \"e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.698080 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.698000 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:53.841425 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.841400 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg"] Jul 23 12:39:53.843863 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:39:53.843840 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54eb1f5d_b743_41f0_bd9c_aac481e400f7.slice/crio-8d3ef63bdf67efebb9827423522ec4f1fe80c3c03d74f48c704735cf9c8e7000 WatchSource:0}: Error finding container 8d3ef63bdf67efebb9827423522ec4f1fe80c3c03d74f48c704735cf9c8e7000: Status 404 returned error can't find the container with id 8d3ef63bdf67efebb9827423522ec4f1fe80c3c03d74f48c704735cf9c8e7000 Jul 23 12:39:53.846145 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.846111 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:39:53.846262 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.846187 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:39:53.846328 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.846269 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:39:53.938387 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.938351 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" event={"ID":"54eb1f5d-b743-41f0-bd9c-aac481e400f7","Type":"ContainerStarted","Data":"3d99cc84f08878f447ccde571b1ecabb6b523de942a22d0debe83c0cc163757a"} Jul 23 12:39:53.938568 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.938411 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" event={"ID":"54eb1f5d-b743-41f0-bd9c-aac481e400f7","Type":"ContainerStarted","Data":"8d3ef63bdf67efebb9827423522ec4f1fe80c3c03d74f48c704735cf9c8e7000"} Jul 23 12:39:53.962714 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:53.962621 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" podStartSLOduration=0.962604166 podStartE2EDuration="962.604166ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:39:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:39:53.962433504 +0000 UTC m=+1805.554968196" watchObservedRunningTime="2026-07-23 12:39:53.962604166 +0000 UTC m=+1805.555138858" Jul 23 12:39:54.698865 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:54.698819 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:54.703779 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:54.703754 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:54.942950 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:54.942917 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:54.943826 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:54.943803 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:39:58.902257 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:39:58.902219 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:40:07.003395 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:07.003359 2578 generic.go:356] "Generic (PLEG): container finished" podID="54eb1f5d-b743-41f0-bd9c-aac481e400f7" containerID="3d99cc84f08878f447ccde571b1ecabb6b523de942a22d0debe83c0cc163757a" exitCode=0 Jul 23 12:40:07.003804 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:07.003412 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" event={"ID":"54eb1f5d-b743-41f0-bd9c-aac481e400f7","Type":"ContainerDied","Data":"3d99cc84f08878f447ccde571b1ecabb6b523de942a22d0debe83c0cc163757a"} Jul 23 12:40:07.003804 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:07.003777 2578 scope.go:117] "RemoveContainer" containerID="3d99cc84f08878f447ccde571b1ecabb6b523de942a22d0debe83c0cc163757a" Jul 23 12:40:07.004788 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:07.004739 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:40:07.004859 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:07.004812 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:40:07.004859 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:07.004843 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:40:08.040294 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:08.040252 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" event={"ID":"54eb1f5d-b743-41f0-bd9c-aac481e400f7","Type":"ContainerStarted","Data":"ac24550af72b52e43a49c3425ea5839d25fb5a1fefde287023b45afff38b5ad7"} Jul 23 12:40:08.698264 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:08.698219 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:40:08.699295 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:08.699271 2578 patch_prober.go:28] interesting pod/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 23 12:40:08.699361 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:08.699323 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" podUID="54eb1f5d-b743-41f0-bd9c-aac481e400f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 23 12:40:09.699448 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:09.699410 2578 patch_prober.go:28] interesting pod/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 23 12:40:09.699893 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:09.699492 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" podUID="54eb1f5d-b743-41f0-bd9c-aac481e400f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 23 12:40:09.943597 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:09.943554 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:40:10.699147 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:10.699109 2578 patch_prober.go:28] interesting pod/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 23 12:40:10.699368 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:10.699163 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" podUID="54eb1f5d-b743-41f0-bd9c-aac481e400f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 23 12:40:11.706726 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:11.706696 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:40:11.707728 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:11.707707 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:40:33.711772 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:33.711733 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:40:35.535647 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:35.535613 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg"] Jul 23 12:40:35.535996 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:35.535869 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" podUID="54eb1f5d-b743-41f0-bd9c-aac481e400f7" containerName="istio-proxy" containerID="cri-o://ac24550af72b52e43a49c3425ea5839d25fb5a1fefde287023b45afff38b5ad7" gracePeriod=30 Jul 23 12:40:40.784092 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.784066 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:40:40.952045 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.952020 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-envoy\") pod \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " Jul 23 12:40:40.952212 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.952059 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-workload-socket\") pod \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " Jul 23 12:40:40.952212 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.952082 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4l9m\" (UniqueName: \"kubernetes.io/projected/54eb1f5d-b743-41f0-bd9c-aac481e400f7-kube-api-access-f4l9m\") pod \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " Jul 23 12:40:40.952212 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.952108 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istiod-ca-cert\") pod \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " Jul 23 12:40:40.952212 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.952177 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-workload-certs\") pod \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " Jul 23 12:40:40.952443 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.952332 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-token\") pod \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " Jul 23 12:40:40.952443 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.952375 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-data\") pod \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " Jul 23 12:40:40.952443 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.952328 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "54eb1f5d-b743-41f0-bd9c-aac481e400f7" (UID: "54eb1f5d-b743-41f0-bd9c-aac481e400f7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:40:40.952443 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.952427 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-credential-socket\") pod \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " Jul 23 12:40:40.952650 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.952450 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-podinfo\") pod \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\" (UID: \"54eb1f5d-b743-41f0-bd9c-aac481e400f7\") " Jul 23 12:40:40.952650 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.952496 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "54eb1f5d-b743-41f0-bd9c-aac481e400f7" (UID: "54eb1f5d-b743-41f0-bd9c-aac481e400f7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:40:40.952752 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.952658 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "54eb1f5d-b743-41f0-bd9c-aac481e400f7" (UID: "54eb1f5d-b743-41f0-bd9c-aac481e400f7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:40:40.952752 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.952681 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "54eb1f5d-b743-41f0-bd9c-aac481e400f7" (UID: "54eb1f5d-b743-41f0-bd9c-aac481e400f7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:40:40.952870 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.952820 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-workload-certs\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:40:40.952870 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.952839 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-credential-socket\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:40:40.952870 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.952854 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-workload-socket\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:40:40.952870 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.952867 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istiod-ca-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:40:40.953129 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.953110 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-data" (OuterVolumeSpecName: "istio-data") pod "54eb1f5d-b743-41f0-bd9c-aac481e400f7" (UID: "54eb1f5d-b743-41f0-bd9c-aac481e400f7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:40:40.954660 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.954629 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "54eb1f5d-b743-41f0-bd9c-aac481e400f7" (UID: "54eb1f5d-b743-41f0-bd9c-aac481e400f7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:40:40.954766 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.954656 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54eb1f5d-b743-41f0-bd9c-aac481e400f7-kube-api-access-f4l9m" (OuterVolumeSpecName: "kube-api-access-f4l9m") pod "54eb1f5d-b743-41f0-bd9c-aac481e400f7" (UID: "54eb1f5d-b743-41f0-bd9c-aac481e400f7"). InnerVolumeSpecName "kube-api-access-f4l9m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:40:40.954766 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.954675 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "54eb1f5d-b743-41f0-bd9c-aac481e400f7" (UID: "54eb1f5d-b743-41f0-bd9c-aac481e400f7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 12:40:40.954766 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:40.954733 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-token" (OuterVolumeSpecName: "istio-token") pod "54eb1f5d-b743-41f0-bd9c-aac481e400f7" (UID: "54eb1f5d-b743-41f0-bd9c-aac481e400f7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:40:41.054057 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:41.054031 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-token\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:40:41.054057 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:41.054052 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-data\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:40:41.054057 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:41.054062 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-podinfo\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:40:41.054270 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:41.054070 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54eb1f5d-b743-41f0-bd9c-aac481e400f7-istio-envoy\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:40:41.054270 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:41.054078 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4l9m\" (UniqueName: \"kubernetes.io/projected/54eb1f5d-b743-41f0-bd9c-aac481e400f7-kube-api-access-f4l9m\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:40:41.201176 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:41.201142 2578 generic.go:356] "Generic (PLEG): container finished" podID="54eb1f5d-b743-41f0-bd9c-aac481e400f7" containerID="ac24550af72b52e43a49c3425ea5839d25fb5a1fefde287023b45afff38b5ad7" exitCode=0 Jul 23 12:40:41.201356 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:41.201188 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" event={"ID":"54eb1f5d-b743-41f0-bd9c-aac481e400f7","Type":"ContainerDied","Data":"ac24550af72b52e43a49c3425ea5839d25fb5a1fefde287023b45afff38b5ad7"} Jul 23 12:40:41.201356 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:41.201245 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" event={"ID":"54eb1f5d-b743-41f0-bd9c-aac481e400f7","Type":"ContainerDied","Data":"8d3ef63bdf67efebb9827423522ec4f1fe80c3c03d74f48c704735cf9c8e7000"} Jul 23 12:40:41.201356 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:41.201267 2578 scope.go:117] "RemoveContainer" containerID="ac24550af72b52e43a49c3425ea5839d25fb5a1fefde287023b45afff38b5ad7" Jul 23 12:40:41.201356 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:41.201187 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg" Jul 23 12:40:41.220075 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:41.220050 2578 scope.go:117] "RemoveContainer" containerID="3d99cc84f08878f447ccde571b1ecabb6b523de942a22d0debe83c0cc163757a" Jul 23 12:40:41.227480 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:41.227464 2578 scope.go:117] "RemoveContainer" containerID="ac24550af72b52e43a49c3425ea5839d25fb5a1fefde287023b45afff38b5ad7" Jul 23 12:40:41.227720 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:40:41.227702 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac24550af72b52e43a49c3425ea5839d25fb5a1fefde287023b45afff38b5ad7\": container with ID starting with ac24550af72b52e43a49c3425ea5839d25fb5a1fefde287023b45afff38b5ad7 not found: ID does not exist" containerID="ac24550af72b52e43a49c3425ea5839d25fb5a1fefde287023b45afff38b5ad7" Jul 23 12:40:41.227779 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:41.227728 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac24550af72b52e43a49c3425ea5839d25fb5a1fefde287023b45afff38b5ad7"} err="failed to get container status \"ac24550af72b52e43a49c3425ea5839d25fb5a1fefde287023b45afff38b5ad7\": rpc error: code = NotFound desc = could not find container \"ac24550af72b52e43a49c3425ea5839d25fb5a1fefde287023b45afff38b5ad7\": container with ID starting with ac24550af72b52e43a49c3425ea5839d25fb5a1fefde287023b45afff38b5ad7 not found: ID does not exist" Jul 23 12:40:41.227779 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:41.227747 2578 scope.go:117] "RemoveContainer" containerID="3d99cc84f08878f447ccde571b1ecabb6b523de942a22d0debe83c0cc163757a" Jul 23 12:40:41.227960 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:40:41.227943 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d99cc84f08878f447ccde571b1ecabb6b523de942a22d0debe83c0cc163757a\": container with ID starting with 3d99cc84f08878f447ccde571b1ecabb6b523de942a22d0debe83c0cc163757a not found: ID does not exist" containerID="3d99cc84f08878f447ccde571b1ecabb6b523de942a22d0debe83c0cc163757a" Jul 23 12:40:41.227999 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:41.227966 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d99cc84f08878f447ccde571b1ecabb6b523de942a22d0debe83c0cc163757a"} err="failed to get container status \"3d99cc84f08878f447ccde571b1ecabb6b523de942a22d0debe83c0cc163757a\": rpc error: code = NotFound desc = could not find container \"3d99cc84f08878f447ccde571b1ecabb6b523de942a22d0debe83c0cc163757a\": container with ID starting with 3d99cc84f08878f447ccde571b1ecabb6b523de942a22d0debe83c0cc163757a not found: ID does not exist" Jul 23 12:40:41.241055 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:41.241028 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg"] Jul 23 12:40:41.242791 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:41.242772 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-3e72e548-openshift-default-f7d6684f5-m6pkg"] Jul 23 12:40:42.935498 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:40:42.935461 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54eb1f5d-b743-41f0-bd9c-aac481e400f7" path="/var/lib/kubelet/pods/54eb1f5d-b743-41f0-bd9c-aac481e400f7/volumes" Jul 23 12:42:09.609512 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:42:09.609475 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:42:20.109734 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:42:20.109699 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:42:31.903213 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:42:31.903151 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:42:51.795778 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:42:51.795688 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:43:21.202280 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.202247 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx"] Jul 23 12:43:21.202676 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.202659 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54eb1f5d-b743-41f0-bd9c-aac481e400f7" containerName="istio-proxy" Jul 23 12:43:21.202676 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.202671 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="54eb1f5d-b743-41f0-bd9c-aac481e400f7" containerName="istio-proxy" Jul 23 12:43:21.202811 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.202735 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="54eb1f5d-b743-41f0-bd9c-aac481e400f7" containerName="istio-proxy" Jul 23 12:43:21.202811 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.202744 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b026a5c-c651-4634-b1ce-fb3e37b84d16" containerName="istio-proxy" Jul 23 12:43:21.202811 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.202751 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="54eb1f5d-b743-41f0-bd9c-aac481e400f7" containerName="istio-proxy" Jul 23 12:43:21.205822 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.205801 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.208494 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.208471 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e67fae86-openshift-default-dockercfg-sqf4b\"" Jul 23 12:43:21.219386 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.219366 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx"] Jul 23 12:43:21.366481 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.366443 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-credential-socket\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.366668 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.366492 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-data\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.366668 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.366580 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istiod-ca-cert\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.366668 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.366638 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26f94\" (UniqueName: \"kubernetes.io/projected/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-kube-api-access-26f94\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.366845 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.366689 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-envoy\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.366845 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.366718 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-workload-certs\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.366845 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.366758 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-workload-socket\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.366845 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.366776 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-podinfo\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.366845 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.366836 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-token\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.467631 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.467527 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-workload-socket\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.467631 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.467588 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-podinfo\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.467866 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.467652 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-token\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.467866 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.467686 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-credential-socket\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.467866 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.467724 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-data\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.467866 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.467767 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istiod-ca-cert\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.467866 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.467816 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26f94\" (UniqueName: \"kubernetes.io/projected/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-kube-api-access-26f94\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.467866 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.467847 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-envoy\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.468169 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.467876 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-workload-certs\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.468169 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.467939 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-workload-socket\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.468169 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.468024 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-credential-socket\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.468169 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.468085 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-data\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.468386 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.468218 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-workload-certs\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.468759 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.468726 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istiod-ca-cert\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.469997 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.469976 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-envoy\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.470300 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.470284 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-podinfo\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.476639 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.476616 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-token\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.476823 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.476806 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26f94\" (UniqueName: \"kubernetes.io/projected/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-kube-api-access-26f94\") pod \"e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.517553 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.517528 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:21.653769 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.653743 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx"] Jul 23 12:43:21.655795 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:43:21.655769 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode942fe07_cb76_4b1c_8cac_df9abbe7f62c.slice/crio-7ba93e9960e6b4119d730de658d4453566f2ba2bdf66432a57e66943e674e196 WatchSource:0}: Error finding container 7ba93e9960e6b4119d730de658d4453566f2ba2bdf66432a57e66943e674e196: Status 404 returned error can't find the container with id 7ba93e9960e6b4119d730de658d4453566f2ba2bdf66432a57e66943e674e196 Jul 23 12:43:21.657930 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.657897 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:43:21.658020 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.657965 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:43:21.658020 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.658001 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:43:21.977662 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.977630 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" event={"ID":"e942fe07-cb76-4b1c-8cac-df9abbe7f62c","Type":"ContainerStarted","Data":"efc5e6032f1c70ca45eab4ed53cc8812c94e51a2416afa417061087cdfce9259"} Jul 23 12:43:21.977866 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.977683 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" event={"ID":"e942fe07-cb76-4b1c-8cac-df9abbe7f62c","Type":"ContainerStarted","Data":"7ba93e9960e6b4119d730de658d4453566f2ba2bdf66432a57e66943e674e196"} Jul 23 12:43:21.999868 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:21.999821 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" podStartSLOduration=0.999803388 podStartE2EDuration="999.803388ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:43:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:43:21.998549403 +0000 UTC m=+2013.591084106" watchObservedRunningTime="2026-07-23 12:43:21.999803388 +0000 UTC m=+2013.592338080" Jul 23 12:43:22.518072 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:22.518040 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:22.522833 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:22.522807 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:22.982845 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:22.982812 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:22.983790 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:22.983770 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:30.803688 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:30.803655 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:43:38.452621 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:38.452585 2578 patch_prober.go:28] interesting pod/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:44182->10.132.0.42:15021: read: connection reset by peer" start-of-body= Jul 23 12:43:38.453095 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:38.452670 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" podUID="e942fe07-cb76-4b1c-8cac-df9abbe7f62c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:44182->10.132.0.42:15021: read: connection reset by peer" Jul 23 12:43:39.062631 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:39.062595 2578 generic.go:356] "Generic (PLEG): container finished" podID="e942fe07-cb76-4b1c-8cac-df9abbe7f62c" containerID="efc5e6032f1c70ca45eab4ed53cc8812c94e51a2416afa417061087cdfce9259" exitCode=0 Jul 23 12:43:39.062840 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:39.062679 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" event={"ID":"e942fe07-cb76-4b1c-8cac-df9abbe7f62c","Type":"ContainerDied","Data":"efc5e6032f1c70ca45eab4ed53cc8812c94e51a2416afa417061087cdfce9259"} Jul 23 12:43:39.063120 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:39.063106 2578 scope.go:117] "RemoveContainer" containerID="efc5e6032f1c70ca45eab4ed53cc8812c94e51a2416afa417061087cdfce9259" Jul 23 12:43:39.064079 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:39.064055 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:43:39.064125 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:39.064115 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:43:39.064166 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:39.064149 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:43:40.069419 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:40.069383 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" event={"ID":"e942fe07-cb76-4b1c-8cac-df9abbe7f62c","Type":"ContainerStarted","Data":"6107605c8236ca65290995ff76a3ac02312ba04d20b2c24c7d2463e848cb3f79"} Jul 23 12:43:40.518395 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:40.518342 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:40.518807 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:40.518776 2578 patch_prober.go:28] interesting pod/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 23 12:43:40.518900 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:40.518837 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" podUID="e942fe07-cb76-4b1c-8cac-df9abbe7f62c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 23 12:43:41.518898 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:41.518854 2578 patch_prober.go:28] interesting pod/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 23 12:43:41.519409 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:41.518930 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" podUID="e942fe07-cb76-4b1c-8cac-df9abbe7f62c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 23 12:43:42.518658 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:42.518157 2578 patch_prober.go:28] interesting pod/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 23 12:43:42.518658 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:42.518260 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" podUID="e942fe07-cb76-4b1c-8cac-df9abbe7f62c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 23 12:43:43.519013 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:43.518970 2578 patch_prober.go:28] interesting pod/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 23 12:43:43.519632 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:43.519601 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" podUID="e942fe07-cb76-4b1c-8cac-df9abbe7f62c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 23 12:43:44.521811 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:44.521776 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:44.522173 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:44.522126 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:44.522826 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:44.522805 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:43:48.114926 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:43:48.114890 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:44:03.694220 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:44:03.694168 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:44:21.498645 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:44:21.498559 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:44:32.602846 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:44:32.602806 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:44:49.010312 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:44:49.010282 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:44:49.017681 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:44:49.017661 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:44:49.079370 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:44:49.079350 2578 scope.go:117] "RemoveContainer" containerID="68b88aebe61ea04820197894ce4a4ac4e464ef7cbcc0a028d3ee9a8debee1c4e" Jul 23 12:45:18.650819 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:18.650787 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx"] Jul 23 12:45:18.651368 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:18.650996 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" podUID="e942fe07-cb76-4b1c-8cac-df9abbe7f62c" containerName="istio-proxy" containerID="cri-o://6107605c8236ca65290995ff76a3ac02312ba04d20b2c24c7d2463e848cb3f79" gracePeriod=30 Jul 23 12:45:22.504982 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:22.504939 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:45:23.901937 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.901913 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:45:23.938931 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.938857 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-envoy\") pod \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " Jul 23 12:45:23.938931 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.938887 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-podinfo\") pod \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " Jul 23 12:45:23.938931 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.938909 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-credential-socket\") pod \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " Jul 23 12:45:23.939224 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.939053 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26f94\" (UniqueName: \"kubernetes.io/projected/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-kube-api-access-26f94\") pod \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " Jul 23 12:45:23.939224 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.939101 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istiod-ca-cert\") pod \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " Jul 23 12:45:23.939224 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.939160 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-workload-certs\") pod \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " Jul 23 12:45:23.939224 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.939207 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e942fe07-cb76-4b1c-8cac-df9abbe7f62c" (UID: "e942fe07-cb76-4b1c-8cac-df9abbe7f62c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:45:23.939442 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.939256 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-data\") pod \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " Jul 23 12:45:23.939442 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.939370 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-workload-socket\") pod \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " Jul 23 12:45:23.939442 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.939405 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-token\") pod \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\" (UID: \"e942fe07-cb76-4b1c-8cac-df9abbe7f62c\") " Jul 23 12:45:23.939442 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.939418 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e942fe07-cb76-4b1c-8cac-df9abbe7f62c" (UID: "e942fe07-cb76-4b1c-8cac-df9abbe7f62c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:45:23.939442 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.939427 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e942fe07-cb76-4b1c-8cac-df9abbe7f62c" (UID: "e942fe07-cb76-4b1c-8cac-df9abbe7f62c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:45:23.939695 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.939578 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e942fe07-cb76-4b1c-8cac-df9abbe7f62c" (UID: "e942fe07-cb76-4b1c-8cac-df9abbe7f62c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:45:23.939814 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.939736 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-credential-socket\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:45:23.939814 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.939755 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istiod-ca-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:45:23.939814 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.939769 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-workload-certs\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:45:23.939814 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.939782 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-workload-socket\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:45:23.940246 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.940225 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-data" (OuterVolumeSpecName: "istio-data") pod "e942fe07-cb76-4b1c-8cac-df9abbe7f62c" (UID: "e942fe07-cb76-4b1c-8cac-df9abbe7f62c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:45:23.941447 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.941421 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e942fe07-cb76-4b1c-8cac-df9abbe7f62c" (UID: "e942fe07-cb76-4b1c-8cac-df9abbe7f62c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 12:45:23.941591 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.941556 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-kube-api-access-26f94" (OuterVolumeSpecName: "kube-api-access-26f94") pod "e942fe07-cb76-4b1c-8cac-df9abbe7f62c" (UID: "e942fe07-cb76-4b1c-8cac-df9abbe7f62c"). InnerVolumeSpecName "kube-api-access-26f94". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:45:23.941888 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.941865 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-token" (OuterVolumeSpecName: "istio-token") pod "e942fe07-cb76-4b1c-8cac-df9abbe7f62c" (UID: "e942fe07-cb76-4b1c-8cac-df9abbe7f62c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:45:23.941962 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:23.941905 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e942fe07-cb76-4b1c-8cac-df9abbe7f62c" (UID: "e942fe07-cb76-4b1c-8cac-df9abbe7f62c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:45:24.040334 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:24.040293 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-data\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:45:24.040334 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:24.040331 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-token\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:45:24.040526 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:24.040343 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-envoy\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:45:24.040526 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:24.040354 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-istio-podinfo\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:45:24.040526 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:24.040368 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26f94\" (UniqueName: \"kubernetes.io/projected/e942fe07-cb76-4b1c-8cac-df9abbe7f62c-kube-api-access-26f94\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:45:24.577989 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:24.577950 2578 generic.go:356] "Generic (PLEG): container finished" podID="e942fe07-cb76-4b1c-8cac-df9abbe7f62c" containerID="6107605c8236ca65290995ff76a3ac02312ba04d20b2c24c7d2463e848cb3f79" exitCode=0 Jul 23 12:45:24.578159 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:24.578001 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" Jul 23 12:45:24.578159 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:24.578033 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" event={"ID":"e942fe07-cb76-4b1c-8cac-df9abbe7f62c","Type":"ContainerDied","Data":"6107605c8236ca65290995ff76a3ac02312ba04d20b2c24c7d2463e848cb3f79"} Jul 23 12:45:24.578159 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:24.578090 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx" event={"ID":"e942fe07-cb76-4b1c-8cac-df9abbe7f62c","Type":"ContainerDied","Data":"7ba93e9960e6b4119d730de658d4453566f2ba2bdf66432a57e66943e674e196"} Jul 23 12:45:24.578159 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:24.578111 2578 scope.go:117] "RemoveContainer" containerID="6107605c8236ca65290995ff76a3ac02312ba04d20b2c24c7d2463e848cb3f79" Jul 23 12:45:24.587347 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:24.587331 2578 scope.go:117] "RemoveContainer" containerID="efc5e6032f1c70ca45eab4ed53cc8812c94e51a2416afa417061087cdfce9259" Jul 23 12:45:24.595109 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:24.595091 2578 scope.go:117] "RemoveContainer" containerID="6107605c8236ca65290995ff76a3ac02312ba04d20b2c24c7d2463e848cb3f79" Jul 23 12:45:24.595358 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:45:24.595339 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6107605c8236ca65290995ff76a3ac02312ba04d20b2c24c7d2463e848cb3f79\": container with ID starting with 6107605c8236ca65290995ff76a3ac02312ba04d20b2c24c7d2463e848cb3f79 not found: ID does not exist" containerID="6107605c8236ca65290995ff76a3ac02312ba04d20b2c24c7d2463e848cb3f79" Jul 23 12:45:24.595414 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:24.595365 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6107605c8236ca65290995ff76a3ac02312ba04d20b2c24c7d2463e848cb3f79"} err="failed to get container status \"6107605c8236ca65290995ff76a3ac02312ba04d20b2c24c7d2463e848cb3f79\": rpc error: code = NotFound desc = could not find container \"6107605c8236ca65290995ff76a3ac02312ba04d20b2c24c7d2463e848cb3f79\": container with ID starting with 6107605c8236ca65290995ff76a3ac02312ba04d20b2c24c7d2463e848cb3f79 not found: ID does not exist" Jul 23 12:45:24.595414 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:24.595382 2578 scope.go:117] "RemoveContainer" containerID="efc5e6032f1c70ca45eab4ed53cc8812c94e51a2416afa417061087cdfce9259" Jul 23 12:45:24.595615 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:45:24.595595 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efc5e6032f1c70ca45eab4ed53cc8812c94e51a2416afa417061087cdfce9259\": container with ID starting with efc5e6032f1c70ca45eab4ed53cc8812c94e51a2416afa417061087cdfce9259 not found: ID does not exist" containerID="efc5e6032f1c70ca45eab4ed53cc8812c94e51a2416afa417061087cdfce9259" Jul 23 12:45:24.595668 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:24.595624 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efc5e6032f1c70ca45eab4ed53cc8812c94e51a2416afa417061087cdfce9259"} err="failed to get container status \"efc5e6032f1c70ca45eab4ed53cc8812c94e51a2416afa417061087cdfce9259\": rpc error: code = NotFound desc = could not find container \"efc5e6032f1c70ca45eab4ed53cc8812c94e51a2416afa417061087cdfce9259\": container with ID starting with efc5e6032f1c70ca45eab4ed53cc8812c94e51a2416afa417061087cdfce9259 not found: ID does not exist" Jul 23 12:45:24.601957 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:24.601936 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx"] Jul 23 12:45:24.608761 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:24.608732 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e67fae86-openshift-default-85fc76f64c-b7rpx"] Jul 23 12:45:24.934743 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:24.934710 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e942fe07-cb76-4b1c-8cac-df9abbe7f62c" path="/var/lib/kubelet/pods/e942fe07-cb76-4b1c-8cac-df9abbe7f62c/volumes" Jul 23 12:45:32.099306 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:45:32.099267 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:46:15.676723 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.676677 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9"] Jul 23 12:46:15.677563 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.677233 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54eb1f5d-b743-41f0-bd9c-aac481e400f7" containerName="istio-proxy" Jul 23 12:46:15.677563 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.677251 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="54eb1f5d-b743-41f0-bd9c-aac481e400f7" containerName="istio-proxy" Jul 23 12:46:15.677563 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.677273 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e942fe07-cb76-4b1c-8cac-df9abbe7f62c" containerName="istio-proxy" Jul 23 12:46:15.677563 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.677282 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e942fe07-cb76-4b1c-8cac-df9abbe7f62c" containerName="istio-proxy" Jul 23 12:46:15.677563 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.677295 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e942fe07-cb76-4b1c-8cac-df9abbe7f62c" containerName="istio-proxy" Jul 23 12:46:15.677563 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.677303 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e942fe07-cb76-4b1c-8cac-df9abbe7f62c" containerName="istio-proxy" Jul 23 12:46:15.677563 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.677385 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e942fe07-cb76-4b1c-8cac-df9abbe7f62c" containerName="istio-proxy" Jul 23 12:46:15.677563 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.677397 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e942fe07-cb76-4b1c-8cac-df9abbe7f62c" containerName="istio-proxy" Jul 23 12:46:15.680776 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.680757 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.683333 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.683314 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c652bb-openshift-default-dockercfg-dfcc8\"" Jul 23 12:46:15.690844 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.690817 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9"] Jul 23 12:46:15.835895 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.835860 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2jb6\" (UniqueName: \"kubernetes.io/projected/db9088b0-9e8e-422a-850b-cd72ade7dfe7-kube-api-access-f2jb6\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.835895 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.835898 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-workload-certs\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.836111 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.835935 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istiod-ca-cert\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.836111 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.835956 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-credential-socket\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.836111 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.835991 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-token\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.836111 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.836087 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-podinfo\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.836260 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.836124 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-workload-socket\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.836260 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.836146 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-data\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.836260 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.836166 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-envoy\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.936893 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.936800 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-podinfo\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.936893 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.936848 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-workload-socket\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.936893 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.936866 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-data\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.936893 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.936886 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-envoy\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.937231 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.936929 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2jb6\" (UniqueName: \"kubernetes.io/projected/db9088b0-9e8e-422a-850b-cd72ade7dfe7-kube-api-access-f2jb6\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.937231 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.936953 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-workload-certs\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.937231 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.937003 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istiod-ca-cert\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.937231 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.937032 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-credential-socket\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.937231 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.937089 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-token\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.937492 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.937295 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-data\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.937492 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.937320 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-workload-socket\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.937587 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.937552 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-workload-certs\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.937645 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.937624 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-credential-socket\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.937848 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.937831 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istiod-ca-cert\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.939734 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.939707 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-envoy\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.939934 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.939917 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-podinfo\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.946439 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.946417 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-token\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.946548 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.946497 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2jb6\" (UniqueName: \"kubernetes.io/projected/db9088b0-9e8e-422a-850b-cd72ade7dfe7-kube-api-access-f2jb6\") pod \"e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:15.993782 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:15.993754 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:16.134541 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:16.134508 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9"] Jul 23 12:46:16.137654 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:46:16.137623 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb9088b0_9e8e_422a_850b_cd72ade7dfe7.slice/crio-897bd648ac3a84c6a03214c5bd679a9bdaf052fbda572eaf3743ec30f21b0a23 WatchSource:0}: Error finding container 897bd648ac3a84c6a03214c5bd679a9bdaf052fbda572eaf3743ec30f21b0a23: Status 404 returned error can't find the container with id 897bd648ac3a84c6a03214c5bd679a9bdaf052fbda572eaf3743ec30f21b0a23 Jul 23 12:46:16.140115 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:16.140079 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:46:16.140182 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:16.140164 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:46:16.140256 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:16.140229 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:46:16.828463 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:16.828426 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" event={"ID":"db9088b0-9e8e-422a-850b-cd72ade7dfe7","Type":"ContainerStarted","Data":"57446c6d54c56528db6d014c460f9f27c213666b04db2958b96eed62ff6ef166"} Jul 23 12:46:16.828938 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:16.828490 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" event={"ID":"db9088b0-9e8e-422a-850b-cd72ade7dfe7","Type":"ContainerStarted","Data":"897bd648ac3a84c6a03214c5bd679a9bdaf052fbda572eaf3743ec30f21b0a23"} Jul 23 12:46:16.854742 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:16.854685 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" podStartSLOduration=1.854667802 podStartE2EDuration="1.854667802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:46:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:46:16.853120634 +0000 UTC m=+2188.445655354" watchObservedRunningTime="2026-07-23 12:46:16.854667802 +0000 UTC m=+2188.447202493" Jul 23 12:46:16.994088 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:16.994046 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:17.999592 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:17.999561 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:18.838432 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:18.838397 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:18.839265 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:18.839244 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:24.502088 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:24.502049 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:46:31.902613 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:31.902566 2578 generic.go:356] "Generic (PLEG): container finished" podID="db9088b0-9e8e-422a-850b-cd72ade7dfe7" containerID="57446c6d54c56528db6d014c460f9f27c213666b04db2958b96eed62ff6ef166" exitCode=0 Jul 23 12:46:31.903018 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:31.902642 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" event={"ID":"db9088b0-9e8e-422a-850b-cd72ade7dfe7","Type":"ContainerDied","Data":"57446c6d54c56528db6d014c460f9f27c213666b04db2958b96eed62ff6ef166"} Jul 23 12:46:31.903062 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:31.903026 2578 scope.go:117] "RemoveContainer" containerID="57446c6d54c56528db6d014c460f9f27c213666b04db2958b96eed62ff6ef166" Jul 23 12:46:31.904085 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:31.904056 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:46:31.904225 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:31.904178 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:46:31.904306 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:31.904269 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:46:32.913525 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:32.913481 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" event={"ID":"db9088b0-9e8e-422a-850b-cd72ade7dfe7","Type":"ContainerStarted","Data":"4833ed41c7640e16339e0cd2f5213b639523597aa71bb9d11097e8d029fee373"} Jul 23 12:46:32.994499 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:32.994461 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:32.995367 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:32.995345 2578 patch_prober.go:28] interesting pod/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 23 12:46:32.995426 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:32.995403 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" podUID="db9088b0-9e8e-422a-850b-cd72ade7dfe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 23 12:46:33.839358 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:33.839318 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:33.994335 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:33.994305 2578 patch_prober.go:28] interesting pod/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 23 12:46:33.994770 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:33.994360 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" podUID="db9088b0-9e8e-422a-850b-cd72ade7dfe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 23 12:46:34.994870 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:34.994833 2578 patch_prober.go:28] interesting pod/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 23 12:46:34.995290 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:34.994897 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" podUID="db9088b0-9e8e-422a-850b-cd72ade7dfe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 23 12:46:35.212939 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:35.207548 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:46:35.994386 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:35.994349 2578 patch_prober.go:28] interesting pod/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 23 12:46:35.994555 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:35.994402 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" podUID="db9088b0-9e8e-422a-850b-cd72ade7dfe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 23 12:46:36.998025 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:36.997992 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:36.998930 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:36.998911 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:46:52.090844 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:46:52.090804 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:47:05.797995 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:47:05.797961 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:47:30.306993 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:47:30.306957 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:47:43.804006 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:47:43.803956 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:48:11.900563 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:11.900521 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:48:27.213925 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:27.213892 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:48:30.630393 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:30.630356 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9"] Jul 23 12:48:30.630801 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:30.630602 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" podUID="db9088b0-9e8e-422a-850b-cd72ade7dfe7" containerName="istio-proxy" containerID="cri-o://4833ed41c7640e16339e0cd2f5213b639523597aa71bb9d11097e8d029fee373" gracePeriod=30 Jul 23 12:48:35.886880 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:35.886844 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:48:36.001068 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.000966 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2jb6\" (UniqueName: \"kubernetes.io/projected/db9088b0-9e8e-422a-850b-cd72ade7dfe7-kube-api-access-f2jb6\") pod \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " Jul 23 12:48:36.001068 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.001037 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-workload-certs\") pod \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " Jul 23 12:48:36.001068 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.001059 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-envoy\") pod \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " Jul 23 12:48:36.001388 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.001087 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-data\") pod \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " Jul 23 12:48:36.001388 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.001122 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-podinfo\") pod \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " Jul 23 12:48:36.001388 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.001172 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istiod-ca-cert\") pod \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " Jul 23 12:48:36.001388 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.001217 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-workload-socket\") pod \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " Jul 23 12:48:36.001388 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.001248 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-credential-socket\") pod \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " Jul 23 12:48:36.001388 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.001304 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-token\") pod \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\" (UID: \"db9088b0-9e8e-422a-850b-cd72ade7dfe7\") " Jul 23 12:48:36.001703 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.001432 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "db9088b0-9e8e-422a-850b-cd72ade7dfe7" (UID: "db9088b0-9e8e-422a-850b-cd72ade7dfe7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:48:36.001703 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.001635 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-workload-certs\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:48:36.001703 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.001649 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "db9088b0-9e8e-422a-850b-cd72ade7dfe7" (UID: "db9088b0-9e8e-422a-850b-cd72ade7dfe7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:48:36.001866 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.001806 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "db9088b0-9e8e-422a-850b-cd72ade7dfe7" (UID: "db9088b0-9e8e-422a-850b-cd72ade7dfe7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:48:36.001866 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.001838 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "db9088b0-9e8e-422a-850b-cd72ade7dfe7" (UID: "db9088b0-9e8e-422a-850b-cd72ade7dfe7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:48:36.002095 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.002075 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-data" (OuterVolumeSpecName: "istio-data") pod "db9088b0-9e8e-422a-850b-cd72ade7dfe7" (UID: "db9088b0-9e8e-422a-850b-cd72ade7dfe7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:48:36.003506 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.003476 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db9088b0-9e8e-422a-850b-cd72ade7dfe7-kube-api-access-f2jb6" (OuterVolumeSpecName: "kube-api-access-f2jb6") pod "db9088b0-9e8e-422a-850b-cd72ade7dfe7" (UID: "db9088b0-9e8e-422a-850b-cd72ade7dfe7"). InnerVolumeSpecName "kube-api-access-f2jb6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:48:36.003506 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.003486 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "db9088b0-9e8e-422a-850b-cd72ade7dfe7" (UID: "db9088b0-9e8e-422a-850b-cd72ade7dfe7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:48:36.003740 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.003720 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-token" (OuterVolumeSpecName: "istio-token") pod "db9088b0-9e8e-422a-850b-cd72ade7dfe7" (UID: "db9088b0-9e8e-422a-850b-cd72ade7dfe7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:48:36.003799 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.003721 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "db9088b0-9e8e-422a-850b-cd72ade7dfe7" (UID: "db9088b0-9e8e-422a-850b-cd72ade7dfe7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 12:48:36.102148 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.102107 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istiod-ca-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:48:36.102148 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.102141 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-workload-socket\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:48:36.102148 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.102151 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-credential-socket\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:48:36.102419 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.102163 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-token\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:48:36.102419 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.102172 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2jb6\" (UniqueName: \"kubernetes.io/projected/db9088b0-9e8e-422a-850b-cd72ade7dfe7-kube-api-access-f2jb6\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:48:36.102419 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.102181 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-envoy\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:48:36.102419 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.102201 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-data\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:48:36.102419 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.102215 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db9088b0-9e8e-422a-850b-cd72ade7dfe7-istio-podinfo\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:48:36.525530 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.525489 2578 generic.go:356] "Generic (PLEG): container finished" podID="db9088b0-9e8e-422a-850b-cd72ade7dfe7" containerID="4833ed41c7640e16339e0cd2f5213b639523597aa71bb9d11097e8d029fee373" exitCode=0 Jul 23 12:48:36.525715 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.525546 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" Jul 23 12:48:36.525715 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.525572 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" event={"ID":"db9088b0-9e8e-422a-850b-cd72ade7dfe7","Type":"ContainerDied","Data":"4833ed41c7640e16339e0cd2f5213b639523597aa71bb9d11097e8d029fee373"} Jul 23 12:48:36.525715 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.525630 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9" event={"ID":"db9088b0-9e8e-422a-850b-cd72ade7dfe7","Type":"ContainerDied","Data":"897bd648ac3a84c6a03214c5bd679a9bdaf052fbda572eaf3743ec30f21b0a23"} Jul 23 12:48:36.525715 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.525652 2578 scope.go:117] "RemoveContainer" containerID="4833ed41c7640e16339e0cd2f5213b639523597aa71bb9d11097e8d029fee373" Jul 23 12:48:36.535932 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.535910 2578 scope.go:117] "RemoveContainer" containerID="57446c6d54c56528db6d014c460f9f27c213666b04db2958b96eed62ff6ef166" Jul 23 12:48:36.547442 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.547368 2578 scope.go:117] "RemoveContainer" containerID="4833ed41c7640e16339e0cd2f5213b639523597aa71bb9d11097e8d029fee373" Jul 23 12:48:36.548287 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:48:36.548253 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4833ed41c7640e16339e0cd2f5213b639523597aa71bb9d11097e8d029fee373\": container with ID starting with 4833ed41c7640e16339e0cd2f5213b639523597aa71bb9d11097e8d029fee373 not found: ID does not exist" containerID="4833ed41c7640e16339e0cd2f5213b639523597aa71bb9d11097e8d029fee373" Jul 23 12:48:36.548402 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.548301 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4833ed41c7640e16339e0cd2f5213b639523597aa71bb9d11097e8d029fee373"} err="failed to get container status \"4833ed41c7640e16339e0cd2f5213b639523597aa71bb9d11097e8d029fee373\": rpc error: code = NotFound desc = could not find container \"4833ed41c7640e16339e0cd2f5213b639523597aa71bb9d11097e8d029fee373\": container with ID starting with 4833ed41c7640e16339e0cd2f5213b639523597aa71bb9d11097e8d029fee373 not found: ID does not exist" Jul 23 12:48:36.548402 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.548329 2578 scope.go:117] "RemoveContainer" containerID="57446c6d54c56528db6d014c460f9f27c213666b04db2958b96eed62ff6ef166" Jul 23 12:48:36.548748 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:48:36.548707 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57446c6d54c56528db6d014c460f9f27c213666b04db2958b96eed62ff6ef166\": container with ID starting with 57446c6d54c56528db6d014c460f9f27c213666b04db2958b96eed62ff6ef166 not found: ID does not exist" containerID="57446c6d54c56528db6d014c460f9f27c213666b04db2958b96eed62ff6ef166" Jul 23 12:48:36.548854 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.548756 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57446c6d54c56528db6d014c460f9f27c213666b04db2958b96eed62ff6ef166"} err="failed to get container status \"57446c6d54c56528db6d014c460f9f27c213666b04db2958b96eed62ff6ef166\": rpc error: code = NotFound desc = could not find container \"57446c6d54c56528db6d014c460f9f27c213666b04db2958b96eed62ff6ef166\": container with ID starting with 57446c6d54c56528db6d014c460f9f27c213666b04db2958b96eed62ff6ef166 not found: ID does not exist" Jul 23 12:48:36.550641 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.550616 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9"] Jul 23 12:48:36.555140 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.555111 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-c652bb-openshift-default-cf4d5977c-8k5q9"] Jul 23 12:48:36.935746 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:36.935713 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db9088b0-9e8e-422a-850b-cd72ade7dfe7" path="/var/lib/kubelet/pods/db9088b0-9e8e-422a-850b-cd72ade7dfe7/volumes" Jul 23 12:48:44.701241 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:44.701142 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:48:53.955936 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:53.955898 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw"] Jul 23 12:48:53.956494 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:53.956472 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db9088b0-9e8e-422a-850b-cd72ade7dfe7" containerName="istio-proxy" Jul 23 12:48:53.956563 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:53.956497 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="db9088b0-9e8e-422a-850b-cd72ade7dfe7" containerName="istio-proxy" Jul 23 12:48:53.956620 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:53.956593 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="db9088b0-9e8e-422a-850b-cd72ade7dfe7" containerName="istio-proxy" Jul 23 12:48:53.956620 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:53.956603 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="db9088b0-9e8e-422a-850b-cd72ade7dfe7" containerName="istio-proxy" Jul 23 12:48:53.962479 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:53.962454 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:53.965392 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:53.965368 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-7dfea2-openshift-default-dockercfg-cs8bx\"" Jul 23 12:48:53.971442 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:53.971419 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw"] Jul 23 12:48:54.080599 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.080561 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-credential-socket\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.080803 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.080607 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-data\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.080803 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.080677 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-workload-socket\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.080803 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.080776 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-podinfo\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.080976 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.080826 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-envoy\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.080976 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.080864 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istiod-ca-cert\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.081076 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.080974 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w6z7\" (UniqueName: \"kubernetes.io/projected/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-kube-api-access-9w6z7\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.081076 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.081027 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-token\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.081076 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.081057 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-workload-certs\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.182525 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.182483 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-workload-certs\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.182718 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.182571 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-credential-socket\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.182718 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.182592 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-data\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.182718 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.182617 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-workload-socket\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.182718 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.182648 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-podinfo\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.182718 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.182685 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-envoy\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.182718 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.182711 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istiod-ca-cert\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.183028 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.182743 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w6z7\" (UniqueName: \"kubernetes.io/projected/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-kube-api-access-9w6z7\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.183028 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.182789 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-token\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.183028 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.182957 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-workload-certs\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.183028 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.182984 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-data\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.183247 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.183098 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-workload-socket\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.183324 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.183305 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-credential-socket\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.183592 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.183574 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istiod-ca-cert\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.185633 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.185606 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-envoy\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.185864 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.185841 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-podinfo\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.192404 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.192374 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w6z7\" (UniqueName: \"kubernetes.io/projected/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-kube-api-access-9w6z7\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.192628 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.192607 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-token\") pod \"e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.275239 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.275135 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:54.416168 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.416139 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw"] Jul 23 12:48:54.418933 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:48:54.418910 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9dbf774_0840_4f4c_adcd_3c27b7af31f2.slice/crio-1bb7e1f9ee1c9a06dbf1f39c65ece1d9da3dbf54b235e29d18ad11a106416b59 WatchSource:0}: Error finding container 1bb7e1f9ee1c9a06dbf1f39c65ece1d9da3dbf54b235e29d18ad11a106416b59: Status 404 returned error can't find the container with id 1bb7e1f9ee1c9a06dbf1f39c65ece1d9da3dbf54b235e29d18ad11a106416b59 Jul 23 12:48:54.421544 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.421507 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:48:54.421655 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.421572 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:48:54.421655 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.421601 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:48:54.612505 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.612467 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" event={"ID":"c9dbf774-0840-4f4c-adcd-3c27b7af31f2","Type":"ContainerStarted","Data":"32f1e597aed2205c51c00041717f80308f1dc14f9f96614a85175864958a4ad4"} Jul 23 12:48:54.612690 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.612527 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" event={"ID":"c9dbf774-0840-4f4c-adcd-3c27b7af31f2","Type":"ContainerStarted","Data":"1bb7e1f9ee1c9a06dbf1f39c65ece1d9da3dbf54b235e29d18ad11a106416b59"} Jul 23 12:48:54.634118 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:54.634066 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" podStartSLOduration=1.634052596 podStartE2EDuration="1.634052596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:48:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:48:54.632263497 +0000 UTC m=+2346.224798191" watchObservedRunningTime="2026-07-23 12:48:54.634052596 +0000 UTC m=+2346.226587288" Jul 23 12:48:55.275917 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:55.275883 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:55.280830 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:55.280806 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:55.617352 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:55.617321 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:55.618238 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:55.618181 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:48:59.707894 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:48:59.707862 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:49:08.690868 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:08.690835 2578 generic.go:356] "Generic (PLEG): container finished" podID="c9dbf774-0840-4f4c-adcd-3c27b7af31f2" containerID="32f1e597aed2205c51c00041717f80308f1dc14f9f96614a85175864958a4ad4" exitCode=0 Jul 23 12:49:08.691354 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:08.690899 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" event={"ID":"c9dbf774-0840-4f4c-adcd-3c27b7af31f2","Type":"ContainerDied","Data":"32f1e597aed2205c51c00041717f80308f1dc14f9f96614a85175864958a4ad4"} Jul 23 12:49:08.691354 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:08.691302 2578 scope.go:117] "RemoveContainer" containerID="32f1e597aed2205c51c00041717f80308f1dc14f9f96614a85175864958a4ad4" Jul 23 12:49:08.692317 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:08.692287 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:49:08.692432 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:08.692358 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:49:08.692432 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:08.692391 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:49:09.697837 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:09.697802 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" event={"ID":"c9dbf774-0840-4f4c-adcd-3c27b7af31f2","Type":"ContainerStarted","Data":"85452e1d2c8fdd328c071b1a2bc421e658828f72dbf99d8901dd7b3a6fb20ab8"} Jul 23 12:49:10.275341 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:10.275303 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:49:10.276334 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:10.276308 2578 patch_prober.go:28] interesting pod/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 23 12:49:10.276394 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:10.276359 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" podUID="c9dbf774-0840-4f4c-adcd-3c27b7af31f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 23 12:49:10.617978 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:10.617888 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:49:11.276490 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:11.276452 2578 patch_prober.go:28] interesting pod/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 23 12:49:11.276991 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:11.276530 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" podUID="c9dbf774-0840-4f4c-adcd-3c27b7af31f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 23 12:49:12.275922 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:12.275883 2578 patch_prober.go:28] interesting pod/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 23 12:49:12.276107 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:12.275952 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" podUID="c9dbf774-0840-4f4c-adcd-3c27b7af31f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 23 12:49:13.275774 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:13.275735 2578 patch_prober.go:28] interesting pod/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 23 12:49:13.276231 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:13.275804 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" podUID="c9dbf774-0840-4f4c-adcd-3c27b7af31f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 23 12:49:13.600783 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:13.600679 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:49:14.280069 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:14.280040 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:49:14.281074 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:14.281051 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:49:36.922713 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:36.922674 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:49:45.116515 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:45.116473 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw"] Jul 23 12:49:45.116956 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:45.116755 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" podUID="c9dbf774-0840-4f4c-adcd-3c27b7af31f2" containerName="istio-proxy" containerID="cri-o://85452e1d2c8fdd328c071b1a2bc421e658828f72dbf99d8901dd7b3a6fb20ab8" gracePeriod=30 Jul 23 12:49:49.041428 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:49.041391 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:49:49.050806 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:49.050782 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:49:50.372625 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.372596 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:49:50.434517 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.434414 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-credential-socket\") pod \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " Jul 23 12:49:50.434517 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.434510 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9w6z7\" (UniqueName: \"kubernetes.io/projected/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-kube-api-access-9w6z7\") pod \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " Jul 23 12:49:50.434728 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.434549 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-envoy\") pod \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " Jul 23 12:49:50.434728 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.434599 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-token\") pod \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " Jul 23 12:49:50.434728 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.434655 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-podinfo\") pod \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " Jul 23 12:49:50.434879 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.434803 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c9dbf774-0840-4f4c-adcd-3c27b7af31f2" (UID: "c9dbf774-0840-4f4c-adcd-3c27b7af31f2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:49:50.435055 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.435032 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-workload-certs\") pod \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " Jul 23 12:49:50.435274 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.435099 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-data\") pod \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " Jul 23 12:49:50.435274 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.435134 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-workload-socket\") pod \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " Jul 23 12:49:50.435274 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.435251 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istiod-ca-cert\") pod \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\" (UID: \"c9dbf774-0840-4f4c-adcd-3c27b7af31f2\") " Jul 23 12:49:50.435274 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.435266 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c9dbf774-0840-4f4c-adcd-3c27b7af31f2" (UID: "c9dbf774-0840-4f4c-adcd-3c27b7af31f2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:49:50.435668 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.435649 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-credential-socket\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:49:50.435782 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.435764 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-workload-certs\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:49:50.435897 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.435722 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c9dbf774-0840-4f4c-adcd-3c27b7af31f2" (UID: "c9dbf774-0840-4f4c-adcd-3c27b7af31f2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:49:50.435961 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.435938 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c9dbf774-0840-4f4c-adcd-3c27b7af31f2" (UID: "c9dbf774-0840-4f4c-adcd-3c27b7af31f2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:49:50.436082 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.436060 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-data" (OuterVolumeSpecName: "istio-data") pod "c9dbf774-0840-4f4c-adcd-3c27b7af31f2" (UID: "c9dbf774-0840-4f4c-adcd-3c27b7af31f2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:49:50.437298 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.437262 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-kube-api-access-9w6z7" (OuterVolumeSpecName: "kube-api-access-9w6z7") pod "c9dbf774-0840-4f4c-adcd-3c27b7af31f2" (UID: "c9dbf774-0840-4f4c-adcd-3c27b7af31f2"). InnerVolumeSpecName "kube-api-access-9w6z7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:49:50.437442 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.437377 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c9dbf774-0840-4f4c-adcd-3c27b7af31f2" (UID: "c9dbf774-0840-4f4c-adcd-3c27b7af31f2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:49:50.437442 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.437380 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-token" (OuterVolumeSpecName: "istio-token") pod "c9dbf774-0840-4f4c-adcd-3c27b7af31f2" (UID: "c9dbf774-0840-4f4c-adcd-3c27b7af31f2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:49:50.437442 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.437377 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c9dbf774-0840-4f4c-adcd-3c27b7af31f2" (UID: "c9dbf774-0840-4f4c-adcd-3c27b7af31f2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 12:49:50.537107 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.537060 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9w6z7\" (UniqueName: \"kubernetes.io/projected/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-kube-api-access-9w6z7\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:49:50.537107 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.537099 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-envoy\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:49:50.537107 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.537113 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-token\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:49:50.537399 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.537124 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-podinfo\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:49:50.537399 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.537135 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istio-data\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:49:50.537399 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.537145 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-workload-socket\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:49:50.537399 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.537156 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9dbf774-0840-4f4c-adcd-3c27b7af31f2-istiod-ca-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:49:50.912464 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.912427 2578 generic.go:356] "Generic (PLEG): container finished" podID="c9dbf774-0840-4f4c-adcd-3c27b7af31f2" containerID="85452e1d2c8fdd328c071b1a2bc421e658828f72dbf99d8901dd7b3a6fb20ab8" exitCode=0 Jul 23 12:49:50.912670 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.912482 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" event={"ID":"c9dbf774-0840-4f4c-adcd-3c27b7af31f2","Type":"ContainerDied","Data":"85452e1d2c8fdd328c071b1a2bc421e658828f72dbf99d8901dd7b3a6fb20ab8"} Jul 23 12:49:50.912670 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.912498 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" Jul 23 12:49:50.912670 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.912555 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw" event={"ID":"c9dbf774-0840-4f4c-adcd-3c27b7af31f2","Type":"ContainerDied","Data":"1bb7e1f9ee1c9a06dbf1f39c65ece1d9da3dbf54b235e29d18ad11a106416b59"} Jul 23 12:49:50.912670 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.912578 2578 scope.go:117] "RemoveContainer" containerID="85452e1d2c8fdd328c071b1a2bc421e658828f72dbf99d8901dd7b3a6fb20ab8" Jul 23 12:49:50.922278 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.922255 2578 scope.go:117] "RemoveContainer" containerID="32f1e597aed2205c51c00041717f80308f1dc14f9f96614a85175864958a4ad4" Jul 23 12:49:50.930254 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.930176 2578 scope.go:117] "RemoveContainer" containerID="85452e1d2c8fdd328c071b1a2bc421e658828f72dbf99d8901dd7b3a6fb20ab8" Jul 23 12:49:50.930550 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:49:50.930526 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85452e1d2c8fdd328c071b1a2bc421e658828f72dbf99d8901dd7b3a6fb20ab8\": container with ID starting with 85452e1d2c8fdd328c071b1a2bc421e658828f72dbf99d8901dd7b3a6fb20ab8 not found: ID does not exist" containerID="85452e1d2c8fdd328c071b1a2bc421e658828f72dbf99d8901dd7b3a6fb20ab8" Jul 23 12:49:50.930658 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.930555 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85452e1d2c8fdd328c071b1a2bc421e658828f72dbf99d8901dd7b3a6fb20ab8"} err="failed to get container status \"85452e1d2c8fdd328c071b1a2bc421e658828f72dbf99d8901dd7b3a6fb20ab8\": rpc error: code = NotFound desc = could not find container \"85452e1d2c8fdd328c071b1a2bc421e658828f72dbf99d8901dd7b3a6fb20ab8\": container with ID starting with 85452e1d2c8fdd328c071b1a2bc421e658828f72dbf99d8901dd7b3a6fb20ab8 not found: ID does not exist" Jul 23 12:49:50.930658 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.930574 2578 scope.go:117] "RemoveContainer" containerID="32f1e597aed2205c51c00041717f80308f1dc14f9f96614a85175864958a4ad4" Jul 23 12:49:50.930787 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:49:50.930771 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32f1e597aed2205c51c00041717f80308f1dc14f9f96614a85175864958a4ad4\": container with ID starting with 32f1e597aed2205c51c00041717f80308f1dc14f9f96614a85175864958a4ad4 not found: ID does not exist" containerID="32f1e597aed2205c51c00041717f80308f1dc14f9f96614a85175864958a4ad4" Jul 23 12:49:50.930825 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.930790 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32f1e597aed2205c51c00041717f80308f1dc14f9f96614a85175864958a4ad4"} err="failed to get container status \"32f1e597aed2205c51c00041717f80308f1dc14f9f96614a85175864958a4ad4\": rpc error: code = NotFound desc = could not find container \"32f1e597aed2205c51c00041717f80308f1dc14f9f96614a85175864958a4ad4\": container with ID starting with 32f1e597aed2205c51c00041717f80308f1dc14f9f96614a85175864958a4ad4 not found: ID does not exist" Jul 23 12:49:50.936516 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.936493 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw"] Jul 23 12:49:50.943340 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:50.943316 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-7dfea2-openshift-default-58985784fc-nljbw"] Jul 23 12:49:52.934662 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:49:52.934630 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9dbf774-0840-4f4c-adcd-3c27b7af31f2" path="/var/lib/kubelet/pods/c9dbf774-0840-4f4c-adcd-3c27b7af31f2/volumes" Jul 23 12:50:05.806903 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:05.806859 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:50:13.146152 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.146068 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9"] Jul 23 12:50:13.146873 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.146815 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db9088b0-9e8e-422a-850b-cd72ade7dfe7" containerName="istio-proxy" Jul 23 12:50:13.146873 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.146837 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="db9088b0-9e8e-422a-850b-cd72ade7dfe7" containerName="istio-proxy" Jul 23 12:50:13.146873 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.146861 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9dbf774-0840-4f4c-adcd-3c27b7af31f2" containerName="istio-proxy" Jul 23 12:50:13.146873 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.146870 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9dbf774-0840-4f4c-adcd-3c27b7af31f2" containerName="istio-proxy" Jul 23 12:50:13.147132 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.146956 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9dbf774-0840-4f4c-adcd-3c27b7af31f2" containerName="istio-proxy" Jul 23 12:50:13.150411 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.150386 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.153147 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.153119 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/949f3ff5-172e-4572-8ae7-54df9baa6636-istiod-ca-cert\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.153288 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.153249 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-token\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.153354 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.153288 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-envoy\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.153408 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.153347 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-d45fc5-openshift-default-dockercfg-r8wph\"" Jul 23 12:50:13.153408 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.153345 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-data\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.153508 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.153423 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-workload-certs\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.153508 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.153456 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-podinfo\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.153588 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.153547 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-workload-socket\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.153588 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.153583 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l64dk\" (UniqueName: \"kubernetes.io/projected/949f3ff5-172e-4572-8ae7-54df9baa6636-kube-api-access-l64dk\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.153666 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.153608 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-credential-socket\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.162081 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.162056 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9"] Jul 23 12:50:13.254349 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.254307 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-envoy\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.254349 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.254355 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-data\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.254625 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.254390 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-workload-certs\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.254625 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.254419 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-podinfo\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.254625 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.254476 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-workload-socket\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.254625 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.254517 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l64dk\" (UniqueName: \"kubernetes.io/projected/949f3ff5-172e-4572-8ae7-54df9baa6636-kube-api-access-l64dk\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.254625 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.254545 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-credential-socket\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.254625 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.254591 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/949f3ff5-172e-4572-8ae7-54df9baa6636-istiod-ca-cert\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.254899 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.254665 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-token\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.254899 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.254799 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-data\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.255942 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.255137 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-workload-socket\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.255942 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.255623 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-workload-certs\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.256160 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.256089 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/949f3ff5-172e-4572-8ae7-54df9baa6636-istiod-ca-cert\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.256279 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.256255 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-credential-socket\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.257980 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.257939 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-podinfo\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.258342 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.258317 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-envoy\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.264321 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.264254 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-token\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.264468 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.264412 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l64dk\" (UniqueName: \"kubernetes.io/projected/949f3ff5-172e-4572-8ae7-54df9baa6636-kube-api-access-l64dk\") pod \"e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.462459 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.462355 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:13.614261 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.614228 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9"] Jul 23 12:50:13.615095 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:50:13.615066 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod949f3ff5_172e_4572_8ae7_54df9baa6636.slice/crio-64a16d3d9616683319f3e1e58bf931154bc908300cddc735b2d92c93408accf6 WatchSource:0}: Error finding container 64a16d3d9616683319f3e1e58bf931154bc908300cddc735b2d92c93408accf6: Status 404 returned error can't find the container with id 64a16d3d9616683319f3e1e58bf931154bc908300cddc735b2d92c93408accf6 Jul 23 12:50:13.617432 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.617394 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:50:13.617542 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.617483 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:50:13.617542 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:13.617513 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:50:14.027965 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:14.027918 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" event={"ID":"949f3ff5-172e-4572-8ae7-54df9baa6636","Type":"ContainerStarted","Data":"4944012ff99c510dc6477d4dc41b480ff705a2d8c57f1b2d49f0471736d1b399"} Jul 23 12:50:14.028183 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:14.027994 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" event={"ID":"949f3ff5-172e-4572-8ae7-54df9baa6636","Type":"ContainerStarted","Data":"64a16d3d9616683319f3e1e58bf931154bc908300cddc735b2d92c93408accf6"} Jul 23 12:50:14.051403 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:14.051349 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" podStartSLOduration=1.051334025 podStartE2EDuration="1.051334025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 12:50:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:50:14.049616782 +0000 UTC m=+2425.642151500" watchObservedRunningTime="2026-07-23 12:50:14.051334025 +0000 UTC m=+2425.643868793" Jul 23 12:50:14.463536 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:14.463496 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:14.468617 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:14.468592 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:15.033061 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:15.033026 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:15.033931 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:15.033910 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:22.795999 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:22.795958 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:50:30.697494 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:30.696613 2578 patch_prober.go:28] interesting pod/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": read tcp 10.132.0.2:36948->10.132.0.45:15021: read: connection reset by peer" start-of-body= Jul 23 12:50:30.697494 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:30.696669 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" podUID="949f3ff5-172e-4572-8ae7-54df9baa6636" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": read tcp 10.132.0.2:36948->10.132.0.45:15021: read: connection reset by peer" Jul 23 12:50:31.137252 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:31.137213 2578 generic.go:356] "Generic (PLEG): container finished" podID="949f3ff5-172e-4572-8ae7-54df9baa6636" containerID="4944012ff99c510dc6477d4dc41b480ff705a2d8c57f1b2d49f0471736d1b399" exitCode=0 Jul 23 12:50:31.137419 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:31.137274 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" event={"ID":"949f3ff5-172e-4572-8ae7-54df9baa6636","Type":"ContainerDied","Data":"4944012ff99c510dc6477d4dc41b480ff705a2d8c57f1b2d49f0471736d1b399"} Jul 23 12:50:31.137709 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:31.137694 2578 scope.go:117] "RemoveContainer" containerID="4944012ff99c510dc6477d4dc41b480ff705a2d8c57f1b2d49f0471736d1b399" Jul 23 12:50:31.138693 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:31.138660 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:50:31.138819 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:31.138743 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:50:31.138819 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:31.138788 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 23 12:50:32.144142 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:32.144104 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" event={"ID":"949f3ff5-172e-4572-8ae7-54df9baa6636","Type":"ContainerStarted","Data":"5a02b30646fa48c5b77ae1b7e4dfe64c1af6871dfbe74278cabae54e058f5ddb"} Jul 23 12:50:32.463084 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:32.462986 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:32.463263 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:32.463186 2578 patch_prober.go:28] interesting pod/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 23 12:50:32.463325 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:32.463303 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" podUID="949f3ff5-172e-4572-8ae7-54df9baa6636" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 23 12:50:33.462981 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:33.462944 2578 patch_prober.go:28] interesting pod/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 23 12:50:33.463413 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:33.463001 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" podUID="949f3ff5-172e-4572-8ae7-54df9baa6636" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 23 12:50:34.463111 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:34.463068 2578 patch_prober.go:28] interesting pod/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 23 12:50:34.463536 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:34.463132 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" podUID="949f3ff5-172e-4572-8ae7-54df9baa6636" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 23 12:50:34.695250 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:34.695216 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:50:35.463250 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:35.463179 2578 patch_prober.go:28] interesting pod/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 23 12:50:35.463714 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:35.463278 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" podUID="949f3ff5-172e-4572-8ae7-54df9baa6636" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 23 12:50:36.467845 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:36.467811 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:36.468300 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:36.468078 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:36.468851 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:36.468830 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:50:45.903860 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:50:45.903824 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:51:08.603914 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:51:08.603880 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:51:25.605487 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:51:25.605443 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:51:44.398211 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:51:44.398120 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:52:00.205281 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:52:00.205242 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:52:20.200685 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:52:20.200645 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:52:37.798016 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:52:37.797974 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:52:56.801805 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:52:56.801767 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:53:12.591663 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:53:12.591585 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:53:27.198858 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:53:27.198815 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:53:34.510934 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:53:34.510904 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:54:19.392625 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:54:19.392586 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:54:32.794160 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:54:32.794119 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:54:49.071306 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:54:49.071238 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:54:49.084857 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:54:49.084832 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jsxrj_024307b2-8735-4d0a-8673-0152c47b31bc/ovn-acl-logging/0.log" Jul 23 12:54:56.697331 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:54:56.697296 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:55:10.492394 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:55:10.492353 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:55:27.598767 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:55:27.598728 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:55:42.600246 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:55:42.600204 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:55:57.793697 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:55:57.793663 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:56:14.702101 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:14.702015 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:56:37.396499 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:37.396462 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:56:46.220707 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:46.220671 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9"] Jul 23 12:56:46.221301 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:46.220938 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" podUID="949f3ff5-172e-4572-8ae7-54df9baa6636" containerName="istio-proxy" containerID="cri-o://5a02b30646fa48c5b77ae1b7e4dfe64c1af6871dfbe74278cabae54e058f5ddb" gracePeriod=30 Jul 23 12:56:51.485284 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.485222 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:56:51.646075 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.646040 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-podinfo\") pod \"949f3ff5-172e-4572-8ae7-54df9baa6636\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " Jul 23 12:56:51.646293 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.646087 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l64dk\" (UniqueName: \"kubernetes.io/projected/949f3ff5-172e-4572-8ae7-54df9baa6636-kube-api-access-l64dk\") pod \"949f3ff5-172e-4572-8ae7-54df9baa6636\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " Jul 23 12:56:51.646293 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.646139 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-workload-socket\") pod \"949f3ff5-172e-4572-8ae7-54df9baa6636\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " Jul 23 12:56:51.646293 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.646168 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-token\") pod \"949f3ff5-172e-4572-8ae7-54df9baa6636\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " Jul 23 12:56:51.646293 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.646250 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-workload-certs\") pod \"949f3ff5-172e-4572-8ae7-54df9baa6636\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " Jul 23 12:56:51.646529 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.646294 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-data\") pod \"949f3ff5-172e-4572-8ae7-54df9baa6636\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " Jul 23 12:56:51.646529 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.646350 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-envoy\") pod \"949f3ff5-172e-4572-8ae7-54df9baa6636\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " Jul 23 12:56:51.646529 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.646388 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/949f3ff5-172e-4572-8ae7-54df9baa6636-istiod-ca-cert\") pod \"949f3ff5-172e-4572-8ae7-54df9baa6636\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " Jul 23 12:56:51.646529 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.646425 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-credential-socket\") pod \"949f3ff5-172e-4572-8ae7-54df9baa6636\" (UID: \"949f3ff5-172e-4572-8ae7-54df9baa6636\") " Jul 23 12:56:51.646529 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.646431 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "949f3ff5-172e-4572-8ae7-54df9baa6636" (UID: "949f3ff5-172e-4572-8ae7-54df9baa6636"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:56:51.646780 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.646537 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "949f3ff5-172e-4572-8ae7-54df9baa6636" (UID: "949f3ff5-172e-4572-8ae7-54df9baa6636"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:56:51.646780 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.646724 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/949f3ff5-172e-4572-8ae7-54df9baa6636-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "949f3ff5-172e-4572-8ae7-54df9baa6636" (UID: "949f3ff5-172e-4572-8ae7-54df9baa6636"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 12:56:51.646780 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.646748 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "949f3ff5-172e-4572-8ae7-54df9baa6636" (UID: "949f3ff5-172e-4572-8ae7-54df9baa6636"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:56:51.646930 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.646826 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-workload-socket\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:56:51.646930 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.646846 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-workload-certs\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:56:51.646930 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.646860 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/949f3ff5-172e-4572-8ae7-54df9baa6636-istiod-ca-cert\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:56:51.646930 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.646872 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-credential-socket\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:56:51.647152 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.647122 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-data" (OuterVolumeSpecName: "istio-data") pod "949f3ff5-172e-4572-8ae7-54df9baa6636" (UID: "949f3ff5-172e-4572-8ae7-54df9baa6636"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:56:51.648450 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.648428 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-token" (OuterVolumeSpecName: "istio-token") pod "949f3ff5-172e-4572-8ae7-54df9baa6636" (UID: "949f3ff5-172e-4572-8ae7-54df9baa6636"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:56:51.648554 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.648452 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/949f3ff5-172e-4572-8ae7-54df9baa6636-kube-api-access-l64dk" (OuterVolumeSpecName: "kube-api-access-l64dk") pod "949f3ff5-172e-4572-8ae7-54df9baa6636" (UID: "949f3ff5-172e-4572-8ae7-54df9baa6636"). InnerVolumeSpecName "kube-api-access-l64dk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 12:56:51.648844 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.648825 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "949f3ff5-172e-4572-8ae7-54df9baa6636" (UID: "949f3ff5-172e-4572-8ae7-54df9baa6636"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 12:56:51.648910 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.648896 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "949f3ff5-172e-4572-8ae7-54df9baa6636" (UID: "949f3ff5-172e-4572-8ae7-54df9baa6636"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 12:56:51.747388 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.747306 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-data\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:56:51.747388 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.747333 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-envoy\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:56:51.747388 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.747344 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-podinfo\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:56:51.747388 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.747356 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l64dk\" (UniqueName: \"kubernetes.io/projected/949f3ff5-172e-4572-8ae7-54df9baa6636-kube-api-access-l64dk\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:56:51.747388 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:51.747369 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/949f3ff5-172e-4572-8ae7-54df9baa6636-istio-token\") on node \"ip-10-0-138-37.ec2.internal\" DevicePath \"\"" Jul 23 12:56:52.028939 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:52.028854 2578 generic.go:356] "Generic (PLEG): container finished" podID="949f3ff5-172e-4572-8ae7-54df9baa6636" containerID="5a02b30646fa48c5b77ae1b7e4dfe64c1af6871dfbe74278cabae54e058f5ddb" exitCode=0 Jul 23 12:56:52.028939 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:52.028905 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" Jul 23 12:56:52.028939 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:52.028919 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" event={"ID":"949f3ff5-172e-4572-8ae7-54df9baa6636","Type":"ContainerDied","Data":"5a02b30646fa48c5b77ae1b7e4dfe64c1af6871dfbe74278cabae54e058f5ddb"} Jul 23 12:56:52.029183 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:52.028972 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" event={"ID":"949f3ff5-172e-4572-8ae7-54df9baa6636","Type":"ContainerDied","Data":"64a16d3d9616683319f3e1e58bf931154bc908300cddc735b2d92c93408accf6"} Jul 23 12:56:52.029183 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:52.029003 2578 scope.go:117] "RemoveContainer" containerID="5a02b30646fa48c5b77ae1b7e4dfe64c1af6871dfbe74278cabae54e058f5ddb" Jul 23 12:56:52.038262 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:52.038247 2578 scope.go:117] "RemoveContainer" containerID="4944012ff99c510dc6477d4dc41b480ff705a2d8c57f1b2d49f0471736d1b399" Jul 23 12:56:52.046079 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:52.046054 2578 scope.go:117] "RemoveContainer" containerID="5a02b30646fa48c5b77ae1b7e4dfe64c1af6871dfbe74278cabae54e058f5ddb" Jul 23 12:56:52.046383 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:56:52.046363 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a02b30646fa48c5b77ae1b7e4dfe64c1af6871dfbe74278cabae54e058f5ddb\": container with ID starting with 5a02b30646fa48c5b77ae1b7e4dfe64c1af6871dfbe74278cabae54e058f5ddb not found: ID does not exist" containerID="5a02b30646fa48c5b77ae1b7e4dfe64c1af6871dfbe74278cabae54e058f5ddb" Jul 23 12:56:52.046459 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:52.046391 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a02b30646fa48c5b77ae1b7e4dfe64c1af6871dfbe74278cabae54e058f5ddb"} err="failed to get container status \"5a02b30646fa48c5b77ae1b7e4dfe64c1af6871dfbe74278cabae54e058f5ddb\": rpc error: code = NotFound desc = could not find container \"5a02b30646fa48c5b77ae1b7e4dfe64c1af6871dfbe74278cabae54e058f5ddb\": container with ID starting with 5a02b30646fa48c5b77ae1b7e4dfe64c1af6871dfbe74278cabae54e058f5ddb not found: ID does not exist" Jul 23 12:56:52.046459 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:52.046410 2578 scope.go:117] "RemoveContainer" containerID="4944012ff99c510dc6477d4dc41b480ff705a2d8c57f1b2d49f0471736d1b399" Jul 23 12:56:52.046660 ip-10-0-138-37 kubenswrapper[2578]: E0723 12:56:52.046641 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4944012ff99c510dc6477d4dc41b480ff705a2d8c57f1b2d49f0471736d1b399\": container with ID starting with 4944012ff99c510dc6477d4dc41b480ff705a2d8c57f1b2d49f0471736d1b399 not found: ID does not exist" containerID="4944012ff99c510dc6477d4dc41b480ff705a2d8c57f1b2d49f0471736d1b399" Jul 23 12:56:52.046707 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:52.046666 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4944012ff99c510dc6477d4dc41b480ff705a2d8c57f1b2d49f0471736d1b399"} err="failed to get container status \"4944012ff99c510dc6477d4dc41b480ff705a2d8c57f1b2d49f0471736d1b399\": rpc error: code = NotFound desc = could not find container \"4944012ff99c510dc6477d4dc41b480ff705a2d8c57f1b2d49f0471736d1b399\": container with ID starting with 4944012ff99c510dc6477d4dc41b480ff705a2d8c57f1b2d49f0471736d1b399 not found: ID does not exist" Jul 23 12:56:52.051854 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:52.051834 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9"] Jul 23 12:56:52.057501 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:52.057478 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9"] Jul 23 12:56:52.469906 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:52.469873 2578 patch_prober.go:28] interesting pod/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 23 12:56:52.470079 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:52.469934 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-d45fc5-openshift-default-54c76d7759-vplq9" podUID="949f3ff5-172e-4572-8ae7-54df9baa6636" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 23 12:56:52.934878 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:56:52.934843 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="949f3ff5-172e-4572-8ae7-54df9baa6636" path="/var/lib/kubelet/pods/949f3ff5-172e-4572-8ae7-54df9baa6636/volumes" Jul 23 12:57:00.000504 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:00.000459 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wxtwj"] Jul 23 12:57:15.751594 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:15.751559 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b_1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8/copy-manifests/0.log" Jul 23 12:57:15.765375 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:15.765347 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b_1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8/manager/0.log" Jul 23 12:57:18.897529 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:18.897494 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-wxtwj_e6654821-3de4-4d0f-891d-7ad9cd0a936c/limitador/0.log" Jul 23 12:57:19.525794 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:19.525763 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-hzjq9_219312fa-7eeb-4989-9032-2da34674f1e4/discovery/0.log" Jul 23 12:57:19.883247 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:19.883217 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mqbl4_7e5e2463-5b11-4730-83fa-29e4b172c94a/istio-proxy/1.log" Jul 23 12:57:20.143999 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:20.143905 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-c5fqg_3a39e593-e5b5-4f09-bb0f-19bc95da799c/payload-processing/0.log" Jul 23 12:57:20.866065 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:20.866034 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-hzwf5_2904f0a7-2a7c-4cef-aef8-dc8f20b12193/main/0.log" Jul 23 12:57:29.807558 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:29.807529 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tcgfq_88f19386-8325-4469-9859-cd876612dcd6/global-pull-secret-syncer/0.log" Jul 23 12:57:29.925349 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:29.925317 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jl6fz_3ee343ae-feea-4448-8ed4-641274b5be7e/konnectivity-agent/0.log" Jul 23 12:57:30.007831 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:30.007799 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-37.ec2.internal_542075cdbfd82dba7e896ceb37827ec4/haproxy/0.log" Jul 23 12:57:35.855326 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:35.855298 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-wxtwj_e6654821-3de4-4d0f-891d-7ad9cd0a936c/limitador/0.log" Jul 23 12:57:37.909855 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:37.909824 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-vmmhc_72799695-c5dc-498b-863e-07240479551c/kube-state-metrics/0.log" Jul 23 12:57:37.935131 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:37.935106 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-vmmhc_72799695-c5dc-498b-863e-07240479551c/kube-rbac-proxy-main/0.log" Jul 23 12:57:37.962622 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:37.962599 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-vmmhc_72799695-c5dc-498b-863e-07240479551c/kube-rbac-proxy-self/0.log" Jul 23 12:57:37.989937 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:37.989909 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-749c994bb4-ppwqt_a7cecab5-988f-4f5c-b8b3-0e6f7bcc6b25/metrics-server/0.log" Jul 23 12:57:38.019576 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:38.019556 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-wnx47_e84909e6-0111-46e7-9ca1-78b5f2cef807/monitoring-plugin/0.log" Jul 23 12:57:38.058043 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:38.058022 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-498sv_0b69a1fe-10ac-4f21-adfc-aad899e19811/node-exporter/0.log" Jul 23 12:57:38.082453 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:38.082432 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-498sv_0b69a1fe-10ac-4f21-adfc-aad899e19811/kube-rbac-proxy/0.log" Jul 23 12:57:38.109147 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:38.109126 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-498sv_0b69a1fe-10ac-4f21-adfc-aad899e19811/init-textfile/0.log" Jul 23 12:57:38.301867 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:38.301798 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-cbzx9_e6f94b8f-cf41-495b-812e-4606666aa268/kube-rbac-proxy-main/0.log" Jul 23 12:57:38.328179 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:38.328158 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-cbzx9_e6f94b8f-cf41-495b-812e-4606666aa268/kube-rbac-proxy-self/0.log" Jul 23 12:57:38.352506 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:38.352488 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-cbzx9_e6f94b8f-cf41-495b-812e-4606666aa268/openshift-state-metrics/0.log" Jul 23 12:57:38.410469 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:38.410449 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67ee23cf-372a-42ce-8db7-9c7eb49d27be/prometheus/0.log" Jul 23 12:57:38.433913 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:38.433892 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67ee23cf-372a-42ce-8db7-9c7eb49d27be/config-reloader/0.log" Jul 23 12:57:38.459266 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:38.459247 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67ee23cf-372a-42ce-8db7-9c7eb49d27be/thanos-sidecar/0.log" Jul 23 12:57:38.484107 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:38.484080 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67ee23cf-372a-42ce-8db7-9c7eb49d27be/kube-rbac-proxy-web/0.log" Jul 23 12:57:38.511693 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:38.511673 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67ee23cf-372a-42ce-8db7-9c7eb49d27be/kube-rbac-proxy/0.log" Jul 23 12:57:38.538723 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:38.538700 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67ee23cf-372a-42ce-8db7-9c7eb49d27be/kube-rbac-proxy-thanos/0.log" Jul 23 12:57:38.565221 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:38.565135 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67ee23cf-372a-42ce-8db7-9c7eb49d27be/init-config-reloader/0.log" Jul 23 12:57:38.650771 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:38.650748 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-gtzbj_9a3f74e9-16f8-40f8-8274-01b7b3031ec6/prometheus-operator-admission-webhook/0.log" Jul 23 12:57:38.680513 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:38.680494 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-644dc889b8-8d7v4_826ef539-e43c-48ac-ae5a-9545d8567801/telemeter-client/0.log" Jul 23 12:57:38.704796 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:38.704775 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-644dc889b8-8d7v4_826ef539-e43c-48ac-ae5a-9545d8567801/reload/0.log" Jul 23 12:57:38.731958 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:38.731939 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-644dc889b8-8d7v4_826ef539-e43c-48ac-ae5a-9545d8567801/kube-rbac-proxy/0.log" Jul 23 12:57:40.296712 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.296671 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm"] Jul 23 12:57:40.297212 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.297177 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9dbf774-0840-4f4c-adcd-3c27b7af31f2" containerName="istio-proxy" Jul 23 12:57:40.297269 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.297216 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9dbf774-0840-4f4c-adcd-3c27b7af31f2" containerName="istio-proxy" Jul 23 12:57:40.297269 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.297229 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="949f3ff5-172e-4572-8ae7-54df9baa6636" containerName="istio-proxy" Jul 23 12:57:40.297269 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.297235 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="949f3ff5-172e-4572-8ae7-54df9baa6636" containerName="istio-proxy" Jul 23 12:57:40.297269 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.297249 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="949f3ff5-172e-4572-8ae7-54df9baa6636" containerName="istio-proxy" Jul 23 12:57:40.297269 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.297254 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="949f3ff5-172e-4572-8ae7-54df9baa6636" containerName="istio-proxy" Jul 23 12:57:40.297427 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.297312 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="949f3ff5-172e-4572-8ae7-54df9baa6636" containerName="istio-proxy" Jul 23 12:57:40.297427 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.297322 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9dbf774-0840-4f4c-adcd-3c27b7af31f2" containerName="istio-proxy" Jul 23 12:57:40.300515 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.300499 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" Jul 23 12:57:40.303098 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.303078 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jmcfw\"/\"default-dockercfg-v244z\"" Jul 23 12:57:40.303962 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.303945 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jmcfw\"/\"kube-root-ca.crt\"" Jul 23 12:57:40.304052 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.304035 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jmcfw\"/\"openshift-service-ca.crt\"" Jul 23 12:57:40.315255 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.315225 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm"] Jul 23 12:57:40.404314 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.404277 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/53b71a8a-c6dc-47eb-99d3-ac15eb8103dd-proc\") pod \"perf-node-gather-daemonset-7q4lm\" (UID: \"53b71a8a-c6dc-47eb-99d3-ac15eb8103dd\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" Jul 23 12:57:40.404498 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.404354 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/53b71a8a-c6dc-47eb-99d3-ac15eb8103dd-sys\") pod \"perf-node-gather-daemonset-7q4lm\" (UID: \"53b71a8a-c6dc-47eb-99d3-ac15eb8103dd\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" Jul 23 12:57:40.404498 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.404376 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/53b71a8a-c6dc-47eb-99d3-ac15eb8103dd-podres\") pod \"perf-node-gather-daemonset-7q4lm\" (UID: \"53b71a8a-c6dc-47eb-99d3-ac15eb8103dd\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" Jul 23 12:57:40.404498 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.404460 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/53b71a8a-c6dc-47eb-99d3-ac15eb8103dd-lib-modules\") pod \"perf-node-gather-daemonset-7q4lm\" (UID: \"53b71a8a-c6dc-47eb-99d3-ac15eb8103dd\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" Jul 23 12:57:40.404602 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.404504 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4p4w\" (UniqueName: \"kubernetes.io/projected/53b71a8a-c6dc-47eb-99d3-ac15eb8103dd-kube-api-access-w4p4w\") pod \"perf-node-gather-daemonset-7q4lm\" (UID: \"53b71a8a-c6dc-47eb-99d3-ac15eb8103dd\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" Jul 23 12:57:40.505896 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.505864 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/53b71a8a-c6dc-47eb-99d3-ac15eb8103dd-sys\") pod \"perf-node-gather-daemonset-7q4lm\" (UID: \"53b71a8a-c6dc-47eb-99d3-ac15eb8103dd\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" Jul 23 12:57:40.506062 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.505904 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/53b71a8a-c6dc-47eb-99d3-ac15eb8103dd-podres\") pod \"perf-node-gather-daemonset-7q4lm\" (UID: \"53b71a8a-c6dc-47eb-99d3-ac15eb8103dd\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" Jul 23 12:57:40.506062 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.505970 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/53b71a8a-c6dc-47eb-99d3-ac15eb8103dd-lib-modules\") pod \"perf-node-gather-daemonset-7q4lm\" (UID: \"53b71a8a-c6dc-47eb-99d3-ac15eb8103dd\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" Jul 23 12:57:40.506062 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.505985 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/53b71a8a-c6dc-47eb-99d3-ac15eb8103dd-sys\") pod \"perf-node-gather-daemonset-7q4lm\" (UID: \"53b71a8a-c6dc-47eb-99d3-ac15eb8103dd\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" Jul 23 12:57:40.506062 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.506003 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4p4w\" (UniqueName: \"kubernetes.io/projected/53b71a8a-c6dc-47eb-99d3-ac15eb8103dd-kube-api-access-w4p4w\") pod \"perf-node-gather-daemonset-7q4lm\" (UID: \"53b71a8a-c6dc-47eb-99d3-ac15eb8103dd\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" Jul 23 12:57:40.506308 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.506102 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/53b71a8a-c6dc-47eb-99d3-ac15eb8103dd-proc\") pod \"perf-node-gather-daemonset-7q4lm\" (UID: \"53b71a8a-c6dc-47eb-99d3-ac15eb8103dd\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" Jul 23 12:57:40.506308 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.506115 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/53b71a8a-c6dc-47eb-99d3-ac15eb8103dd-lib-modules\") pod \"perf-node-gather-daemonset-7q4lm\" (UID: \"53b71a8a-c6dc-47eb-99d3-ac15eb8103dd\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" Jul 23 12:57:40.506308 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.506132 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/53b71a8a-c6dc-47eb-99d3-ac15eb8103dd-podres\") pod \"perf-node-gather-daemonset-7q4lm\" (UID: \"53b71a8a-c6dc-47eb-99d3-ac15eb8103dd\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" Jul 23 12:57:40.506308 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.506226 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/53b71a8a-c6dc-47eb-99d3-ac15eb8103dd-proc\") pod \"perf-node-gather-daemonset-7q4lm\" (UID: \"53b71a8a-c6dc-47eb-99d3-ac15eb8103dd\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" Jul 23 12:57:40.514528 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.514505 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4p4w\" (UniqueName: \"kubernetes.io/projected/53b71a8a-c6dc-47eb-99d3-ac15eb8103dd-kube-api-access-w4p4w\") pod \"perf-node-gather-daemonset-7q4lm\" (UID: \"53b71a8a-c6dc-47eb-99d3-ac15eb8103dd\") " pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" Jul 23 12:57:40.612158 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.612129 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" Jul 23 12:57:40.742743 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:40.742641 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm"] Jul 23 12:57:40.745333 ip-10-0-138-37 kubenswrapper[2578]: W0723 12:57:40.745303 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod53b71a8a_c6dc_47eb_99d3_ac15eb8103dd.slice/crio-bd6d21ae89390c6bc01d939d1bc8d31e1e4376ecb0a0c0a6c47256d3980ba435 WatchSource:0}: Error finding container bd6d21ae89390c6bc01d939d1bc8d31e1e4376ecb0a0c0a6c47256d3980ba435: Status 404 returned error can't find the container with id bd6d21ae89390c6bc01d939d1bc8d31e1e4376ecb0a0c0a6c47256d3980ba435 Jul 23 12:57:41.277504 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:41.277465 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" event={"ID":"53b71a8a-c6dc-47eb-99d3-ac15eb8103dd","Type":"ContainerStarted","Data":"af0f6ce3bcd17a21ca1c3a76d976df95aa2e4b016cf223327deb08d5e4c01288"} Jul 23 12:57:41.277710 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:41.277539 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" Jul 23 12:57:41.277710 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:41.277568 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" event={"ID":"53b71a8a-c6dc-47eb-99d3-ac15eb8103dd","Type":"ContainerStarted","Data":"bd6d21ae89390c6bc01d939d1bc8d31e1e4376ecb0a0c0a6c47256d3980ba435"} Jul 23 12:57:41.282066 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:41.282043 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7d76c58cfd-7ncbt_b4ecfe4d-2076-49e4-b478-bf544b78c175/console/0.log" Jul 23 12:57:41.296659 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:41.296611 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" podStartSLOduration=1.296594408 podStartE2EDuration="1.296594408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 12:57:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 12:57:41.295006598 +0000 UTC m=+2872.887541292" watchObservedRunningTime="2026-07-23 12:57:41.296594408 +0000 UTC m=+2872.889129121" Jul 23 12:57:41.320222 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:41.320187 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-5zqgt_e02a2643-257b-4aef-a900-3528d209b039/download-server/0.log" Jul 23 12:57:41.937646 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:41.937613 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-g68ch_034d97e3-9d7e-4957-8235-25b1eb22eb85/volume-data-source-validator/0.log" Jul 23 12:57:43.011929 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:43.011902 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2wl6p_f8597295-ae1a-42e6-ac3f-293915f8a69c/dns/0.log" Jul 23 12:57:43.035470 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:43.035445 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2wl6p_f8597295-ae1a-42e6-ac3f-293915f8a69c/kube-rbac-proxy/0.log" Jul 23 12:57:43.212241 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:43.212215 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rj4l2_d3c0c881-554f-4572-bdaa-c0ff060c50ec/dns-node-resolver/0.log" Jul 23 12:57:43.838402 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:43.838376 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zdc7h_374f6cfe-ca3e-424f-9875-3040a3e78447/node-ca/0.log" Jul 23 12:57:45.078104 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:45.078070 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-hzjq9_219312fa-7eeb-4989-9032-2da34674f1e4/discovery/0.log" Jul 23 12:57:45.159526 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:45.159487 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mqbl4_7e5e2463-5b11-4730-83fa-29e4b172c94a/istio-proxy/0.log" Jul 23 12:57:45.184598 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:45.184571 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mqbl4_7e5e2463-5b11-4730-83fa-29e4b172c94a/istio-proxy/1.log" Jul 23 12:57:45.250719 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:45.250689 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-c5fqg_3a39e593-e5b5-4f09-bb0f-19bc95da799c/payload-processing/0.log" Jul 23 12:57:45.998598 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:45.998563 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ffzsb_d13b8306-c63d-4eaa-92be-8ba79051dbd3/serve-healthcheck-canary/0.log" Jul 23 12:57:46.499429 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:46.499404 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-8nqgf_4eaee556-6e0b-46ae-9af7-da2785b3e34a/insights-operator/0.log" Jul 23 12:57:46.501167 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:46.501148 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-8nqgf_4eaee556-6e0b-46ae-9af7-da2785b3e34a/insights-operator/1.log" Jul 23 12:57:46.526643 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:46.526615 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7z6nv_09f3adc9-0ded-4260-bedb-cf97199c0a39/kube-rbac-proxy/0.log" Jul 23 12:57:46.552407 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:46.552379 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7z6nv_09f3adc9-0ded-4260-bedb-cf97199c0a39/exporter/0.log" Jul 23 12:57:46.578203 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:46.578171 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7z6nv_09f3adc9-0ded-4260-bedb-cf97199c0a39/extractor/0.log" Jul 23 12:57:47.292328 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:47.292301 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jmcfw/perf-node-gather-daemonset-7q4lm" Jul 23 12:57:49.459740 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:49.459657 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b_1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8/manager/0.log" Jul 23 12:57:49.479982 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:49.479960 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d6ddd8c6d-mbc6b_1dc2e72c-6f6b-40ed-845b-d7f4dbc5c3a8/copy-manifests/0.log" Jul 23 12:57:50.878114 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:50.878083 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-hzjq9_219312fa-7eeb-4989-9032-2da34674f1e4/discovery/0.log" Jul 23 12:57:50.933812 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:50.933782 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mqbl4_7e5e2463-5b11-4730-83fa-29e4b172c94a/istio-proxy/0.log" Jul 23 12:57:50.955474 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:50.955450 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mqbl4_7e5e2463-5b11-4730-83fa-29e4b172c94a/istio-proxy/1.log" Jul 23 12:57:50.958327 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:50.958309 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7c76dc7d5f-ctjfq_3a4b3135-dfbc-44a4-8753-acdd19322d4d/manager/0.log" Jul 23 12:57:50.996168 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:50.996143 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-kqrpf_a9c27a26-2071-4c16-b71d-fb7f81ee61c4/openshift-lws-operator/0.log" Jul 23 12:57:50.999760 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:50.999731 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-c5fqg_3a39e593-e5b5-4f09-bb0f-19bc95da799c/payload-processing/0.log" Jul 23 12:57:52.412726 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:52.412694 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-hzjq9_219312fa-7eeb-4989-9032-2da34674f1e4/discovery/0.log" Jul 23 12:57:52.466818 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:52.466793 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mqbl4_7e5e2463-5b11-4730-83fa-29e4b172c94a/istio-proxy/0.log" Jul 23 12:57:52.491565 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:52.491541 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mqbl4_7e5e2463-5b11-4730-83fa-29e4b172c94a/istio-proxy/1.log" Jul 23 12:57:52.535247 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:52.535223 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-c5fqg_3a39e593-e5b5-4f09-bb0f-19bc95da799c/payload-processing/0.log" Jul 23 12:57:56.030349 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:56.030325 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-lh8jx_3020a32a-dfd8-4f43-94ea-bb675c26f0fe/migrator/0.log" Jul 23 12:57:56.055726 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:56.055695 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-lh8jx_3020a32a-dfd8-4f43-94ea-bb675c26f0fe/graceful-termination/0.log" Jul 23 12:57:56.444099 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:56.444064 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-cns49_fafa146f-b22a-49a7-b7c6-263e59d7d9ab/kube-storage-version-migrator-operator/1.log" Jul 23 12:57:56.444993 ip-10-0-138-37 kubenswrapper[2578]: I0723 12:57:56.444973 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-cns49_fafa146f-b22a-49a7-b7c6-263e59d7d9ab/kube-storage-version-migrator-operator/0.log"