Jul 24 14:54:06.937401 ip-10-0-137-108 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 24 14:54:06.937413 ip-10-0-137-108 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 24 14:54:06.937423 ip-10-0-137-108 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 24 14:54:06.937749 ip-10-0-137-108 systemd[1]: Failed to start Kubernetes Kubelet. Jul 24 14:54:17.015892 ip-10-0-137-108 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Jul 24 14:54:17.015907 ip-10-0-137-108 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 13e31eef3f744dcf94769f8be0bd965e -- Jul 24 14:56:07.657188 ip-10-0-137-108 systemd[1]: Starting Kubernetes Kubelet... Jul 24 14:56:08.048768 ip-10-0-137-108 kubenswrapper[2531]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 14:56:08.048768 ip-10-0-137-108 kubenswrapper[2531]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 24 14:56:08.048768 ip-10-0-137-108 kubenswrapper[2531]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 14:56:08.048768 ip-10-0-137-108 kubenswrapper[2531]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 24 14:56:08.048768 ip-10-0-137-108 kubenswrapper[2531]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 14:56:08.049639 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.049530 2531 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 24 14:56:08.054997 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.054976 2531 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 14:56:08.054997 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.054994 2531 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 14:56:08.054997 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.054999 2531 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 14:56:08.054997 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055003 2531 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 14:56:08.055161 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055006 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 14:56:08.055161 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055009 2531 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 14:56:08.055161 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055013 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 14:56:08.055161 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055016 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 14:56:08.055161 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055019 2531 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 14:56:08.055161 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055021 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 14:56:08.055161 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055024 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 14:56:08.055161 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055027 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 14:56:08.055161 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055029 2531 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 14:56:08.055161 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055032 2531 feature_gate.go:328] unrecognized feature gate: Example Jul 24 14:56:08.055161 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055034 2531 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 14:56:08.055161 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055038 2531 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 14:56:08.055161 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055040 2531 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 14:56:08.055161 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055042 2531 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 14:56:08.055161 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055045 2531 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 14:56:08.055161 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055048 2531 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 14:56:08.055161 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055050 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 14:56:08.055161 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055053 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 14:56:08.055161 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055055 2531 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 14:56:08.055161 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055059 2531 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 14:56:08.055659 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055061 2531 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 14:56:08.055659 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055064 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 14:56:08.055659 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055068 2531 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 14:56:08.055659 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055072 2531 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 14:56:08.055659 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055074 2531 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 14:56:08.055659 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055077 2531 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 14:56:08.055659 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055080 2531 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 14:56:08.055659 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055082 2531 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 14:56:08.055659 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055086 2531 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 14:56:08.055659 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055088 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 14:56:08.055659 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055090 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 14:56:08.055659 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055093 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 14:56:08.055659 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055096 2531 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 14:56:08.055659 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055099 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 14:56:08.055659 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055102 2531 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 14:56:08.055659 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055105 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 14:56:08.055659 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055107 2531 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 14:56:08.055659 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055110 2531 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 14:56:08.055659 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055113 2531 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 14:56:08.055659 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055115 2531 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 14:56:08.056145 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055118 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 14:56:08.056145 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055120 2531 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 14:56:08.056145 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055123 2531 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 14:56:08.056145 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055125 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 14:56:08.056145 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055128 2531 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 14:56:08.056145 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055130 2531 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 14:56:08.056145 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055133 2531 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 14:56:08.056145 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055136 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 14:56:08.056145 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055138 2531 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 14:56:08.056145 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055141 2531 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 14:56:08.056145 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055143 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 14:56:08.056145 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055146 2531 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 14:56:08.056145 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055148 2531 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 14:56:08.056145 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055151 2531 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 14:56:08.056145 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055153 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 14:56:08.056145 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055155 2531 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 14:56:08.056145 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055158 2531 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 14:56:08.056145 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055160 2531 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 14:56:08.056145 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055163 2531 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 14:56:08.056145 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055165 2531 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 14:56:08.056628 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055168 2531 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 14:56:08.056628 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055170 2531 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 14:56:08.056628 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055173 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 14:56:08.056628 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055176 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 14:56:08.056628 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055179 2531 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 14:56:08.056628 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055182 2531 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 14:56:08.056628 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055185 2531 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 14:56:08.056628 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055187 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 14:56:08.056628 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055190 2531 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 14:56:08.056628 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055193 2531 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 14:56:08.056628 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055195 2531 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 14:56:08.056628 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055199 2531 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 14:56:08.056628 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055202 2531 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 14:56:08.056628 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055204 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 14:56:08.056628 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055208 2531 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 14:56:08.056628 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055211 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 14:56:08.056628 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055214 2531 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 14:56:08.056628 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055217 2531 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 14:56:08.056628 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055220 2531 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 14:56:08.056628 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055223 2531 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 14:56:08.057132 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055226 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 14:56:08.057132 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055229 2531 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 14:56:08.057132 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055232 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 14:56:08.057132 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055234 2531 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 14:56:08.057132 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055237 2531 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 14:56:08.057132 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055239 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 14:56:08.057132 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055242 2531 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 14:56:08.057132 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055244 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 14:56:08.057132 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055249 2531 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 14:56:08.057132 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055253 2531 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 14:56:08.057132 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055256 2531 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 14:56:08.057132 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055259 2531 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 14:56:08.057132 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055262 2531 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 14:56:08.057132 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055265 2531 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 14:56:08.057132 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055268 2531 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 14:56:08.057132 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055986 2531 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 14:56:08.057132 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055993 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 14:56:08.057132 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055996 2531 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 14:56:08.057554 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.055999 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 14:56:08.057554 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056002 2531 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 14:56:08.057554 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056004 2531 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 14:56:08.057554 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056009 2531 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 14:56:08.057554 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056013 2531 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 14:56:08.057554 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056016 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 14:56:08.057554 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056019 2531 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 14:56:08.057554 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056022 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 14:56:08.057554 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056025 2531 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 14:56:08.057554 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056028 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 14:56:08.057554 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056031 2531 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 14:56:08.057554 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056033 2531 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 14:56:08.057554 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056036 2531 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 14:56:08.057554 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056039 2531 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 14:56:08.057554 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056043 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 14:56:08.057554 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056046 2531 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 14:56:08.057554 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056048 2531 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 14:56:08.057554 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056051 2531 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 14:56:08.057554 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056053 2531 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 14:56:08.058074 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056056 2531 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 14:56:08.058074 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056058 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 14:56:08.058074 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056060 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 14:56:08.058074 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056063 2531 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 14:56:08.058074 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056066 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 14:56:08.058074 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056069 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 14:56:08.058074 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056071 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 14:56:08.058074 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056073 2531 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 14:56:08.058074 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056076 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 14:56:08.058074 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056078 2531 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 14:56:08.058074 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056082 2531 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 14:56:08.058074 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056085 2531 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 14:56:08.058074 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056089 2531 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 14:56:08.058074 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056093 2531 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 14:56:08.058074 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056095 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 14:56:08.058074 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056098 2531 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 14:56:08.058074 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056100 2531 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 14:56:08.058074 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056105 2531 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 14:56:08.058074 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056108 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 14:56:08.058536 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056111 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 14:56:08.058536 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056114 2531 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 14:56:08.058536 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056117 2531 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 14:56:08.058536 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056119 2531 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 14:56:08.058536 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056122 2531 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 14:56:08.058536 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056125 2531 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 14:56:08.058536 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056128 2531 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 14:56:08.058536 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056131 2531 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 14:56:08.058536 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056134 2531 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 14:56:08.058536 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056137 2531 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 14:56:08.058536 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056139 2531 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 14:56:08.058536 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056141 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 14:56:08.058536 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056144 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 14:56:08.058536 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056146 2531 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 14:56:08.058536 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056150 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 14:56:08.058536 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056152 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 14:56:08.058536 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056155 2531 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 14:56:08.058536 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056157 2531 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 14:56:08.058536 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056160 2531 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 14:56:08.058536 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056163 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 14:56:08.059046 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056165 2531 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 14:56:08.059046 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056168 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 14:56:08.059046 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056170 2531 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 14:56:08.059046 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056172 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 14:56:08.059046 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056175 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 14:56:08.059046 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056178 2531 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 14:56:08.059046 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056181 2531 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 14:56:08.059046 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056185 2531 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 14:56:08.059046 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056187 2531 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 14:56:08.059046 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056190 2531 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 14:56:08.059046 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056192 2531 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 14:56:08.059046 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056195 2531 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 14:56:08.059046 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056197 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 14:56:08.059046 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056199 2531 feature_gate.go:328] unrecognized feature gate: Example Jul 24 14:56:08.059046 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056202 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 14:56:08.059046 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056204 2531 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 14:56:08.059046 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056207 2531 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 14:56:08.059046 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056209 2531 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 14:56:08.059046 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056211 2531 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 14:56:08.059046 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056214 2531 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 14:56:08.059525 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056217 2531 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 14:56:08.059525 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056219 2531 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 14:56:08.059525 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056222 2531 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 14:56:08.059525 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056225 2531 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 14:56:08.059525 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056228 2531 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 14:56:08.059525 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056230 2531 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 14:56:08.059525 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056233 2531 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 14:56:08.059525 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056235 2531 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 14:56:08.059525 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056238 2531 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 14:56:08.059525 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056240 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 14:56:08.059525 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056243 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 14:56:08.059525 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056245 2531 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 14:56:08.059525 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056248 2531 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 14:56:08.059525 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056250 2531 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 14:56:08.059525 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056252 2531 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 14:56:08.059525 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056255 2531 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 14:56:08.059525 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056257 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 14:56:08.059525 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056260 2531 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 14:56:08.059525 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056607 2531 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 14:56:08.059525 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056611 2531 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 14:56:08.060020 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056614 2531 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 14:56:08.060020 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056617 2531 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 14:56:08.060020 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056619 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 14:56:08.060020 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056622 2531 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 14:56:08.060020 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056624 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 14:56:08.060020 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056627 2531 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 14:56:08.060020 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056630 2531 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 14:56:08.060020 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056633 2531 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 14:56:08.060020 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056638 2531 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 14:56:08.060020 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056642 2531 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 14:56:08.060020 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056645 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 14:56:08.060020 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056648 2531 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 14:56:08.060020 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056651 2531 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 14:56:08.060020 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056654 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 14:56:08.060020 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056657 2531 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 14:56:08.060020 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056660 2531 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 14:56:08.060020 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056663 2531 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 14:56:08.060020 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056665 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 14:56:08.060020 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056668 2531 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 14:56:08.060020 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056671 2531 feature_gate.go:328] unrecognized feature gate: Example Jul 24 14:56:08.060499 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056673 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 14:56:08.060499 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056676 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 14:56:08.060499 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056679 2531 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 14:56:08.060499 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056682 2531 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 14:56:08.060499 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056685 2531 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 14:56:08.060499 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056687 2531 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 14:56:08.060499 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056690 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 14:56:08.060499 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056693 2531 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 14:56:08.060499 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056695 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 14:56:08.060499 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056717 2531 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 14:56:08.060499 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056721 2531 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 14:56:08.060499 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056724 2531 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 14:56:08.060499 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056727 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 14:56:08.060499 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056729 2531 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 14:56:08.060499 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056732 2531 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 14:56:08.060499 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056734 2531 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 14:56:08.060499 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056737 2531 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 14:56:08.060499 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056740 2531 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 14:56:08.060499 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056742 2531 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 14:56:08.060975 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056746 2531 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 14:56:08.060975 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056749 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 14:56:08.060975 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056752 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 14:56:08.060975 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056755 2531 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 14:56:08.060975 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056759 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 14:56:08.060975 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056761 2531 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 14:56:08.060975 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056764 2531 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 14:56:08.060975 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056768 2531 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 14:56:08.060975 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056772 2531 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 14:56:08.060975 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056775 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 14:56:08.060975 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056777 2531 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 14:56:08.060975 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056780 2531 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 14:56:08.060975 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056782 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 14:56:08.060975 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056784 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 14:56:08.060975 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056787 2531 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 14:56:08.060975 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056790 2531 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 14:56:08.060975 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056792 2531 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 14:56:08.060975 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056794 2531 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 14:56:08.060975 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056797 2531 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 14:56:08.061438 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056800 2531 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 14:56:08.061438 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056802 2531 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 14:56:08.061438 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056805 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 14:56:08.061438 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056808 2531 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 14:56:08.061438 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056811 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 14:56:08.061438 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056813 2531 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 14:56:08.061438 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056816 2531 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 14:56:08.061438 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056818 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 14:56:08.061438 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056820 2531 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 14:56:08.061438 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056823 2531 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 14:56:08.061438 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056825 2531 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 14:56:08.061438 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056828 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 14:56:08.061438 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056830 2531 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 14:56:08.061438 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056832 2531 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 14:56:08.061438 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056835 2531 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 14:56:08.061438 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056837 2531 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 14:56:08.061438 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056840 2531 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 14:56:08.061438 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056842 2531 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 14:56:08.061438 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056845 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 14:56:08.061438 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056847 2531 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 14:56:08.061941 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056850 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 14:56:08.061941 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056852 2531 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 14:56:08.061941 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056855 2531 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 14:56:08.061941 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056858 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 14:56:08.061941 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056878 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 14:56:08.061941 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056883 2531 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 14:56:08.061941 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056886 2531 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 14:56:08.061941 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056889 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 14:56:08.061941 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056891 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 14:56:08.061941 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056894 2531 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 14:56:08.061941 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056897 2531 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 14:56:08.061941 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056901 2531 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 14:56:08.061941 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056903 2531 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 14:56:08.061941 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056906 2531 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 14:56:08.061941 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056910 2531 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 14:56:08.061941 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056913 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 14:56:08.061941 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056916 2531 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 14:56:08.061941 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056918 2531 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 14:56:08.061941 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.056921 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.056983 2531 flags.go:64] FLAG: --address="0.0.0.0" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.056990 2531 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.056995 2531 flags.go:64] FLAG: --anonymous-auth="true" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.056999 2531 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057003 2531 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057006 2531 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057010 2531 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057014 2531 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057017 2531 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057020 2531 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057026 2531 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057029 2531 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057032 2531 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057035 2531 flags.go:64] FLAG: --cgroup-root="" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057038 2531 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057041 2531 flags.go:64] FLAG: --client-ca-file="" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057045 2531 flags.go:64] FLAG: --cloud-provider="external" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057048 2531 flags.go:64] FLAG: --cluster-dns="[]" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057052 2531 flags.go:64] FLAG: --cluster-domain="" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057054 2531 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057057 2531 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057060 2531 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057063 2531 flags.go:64] FLAG: --container-log-max-files="5" Jul 24 14:56:08.062401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057067 2531 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057070 2531 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057073 2531 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057076 2531 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057080 2531 flags.go:64] FLAG: --contention-profiling="false" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057083 2531 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057087 2531 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057090 2531 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057092 2531 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057096 2531 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057099 2531 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057102 2531 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057105 2531 flags.go:64] FLAG: --enable-load-reader="false" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057107 2531 flags.go:64] FLAG: --enable-server="true" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057110 2531 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057114 2531 flags.go:64] FLAG: --event-burst="100" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057117 2531 flags.go:64] FLAG: --event-qps="50" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057120 2531 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057122 2531 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057126 2531 flags.go:64] FLAG: --eviction-hard="" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057130 2531 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057132 2531 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057136 2531 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057139 2531 flags.go:64] FLAG: --eviction-soft="" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057142 2531 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 24 14:56:08.062982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057144 2531 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057162 2531 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057165 2531 flags.go:64] FLAG: --experimental-mounter-path="" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057168 2531 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057171 2531 flags.go:64] FLAG: --fail-swap-on="true" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057174 2531 flags.go:64] FLAG: --feature-gates="" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057185 2531 flags.go:64] FLAG: --file-check-frequency="20s" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057188 2531 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057191 2531 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057194 2531 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057197 2531 flags.go:64] FLAG: --healthz-port="10248" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057200 2531 flags.go:64] FLAG: --help="false" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057203 2531 flags.go:64] FLAG: --hostname-override="ip-10-0-137-108.ec2.internal" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057206 2531 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057209 2531 flags.go:64] FLAG: --http-check-frequency="20s" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057212 2531 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057215 2531 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057218 2531 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057221 2531 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057223 2531 flags.go:64] FLAG: --image-service-endpoint="" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057226 2531 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057229 2531 flags.go:64] FLAG: --kube-api-burst="100" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057231 2531 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057234 2531 flags.go:64] FLAG: --kube-api-qps="50" Jul 24 14:56:08.063622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057237 2531 flags.go:64] FLAG: --kube-reserved="" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057239 2531 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057243 2531 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057246 2531 flags.go:64] FLAG: --kubelet-cgroups="" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057249 2531 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057252 2531 flags.go:64] FLAG: --lock-file="" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057254 2531 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057257 2531 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057260 2531 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057265 2531 flags.go:64] FLAG: --log-json-split-stream="false" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057268 2531 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057271 2531 flags.go:64] FLAG: --log-text-split-stream="false" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057274 2531 flags.go:64] FLAG: --logging-format="text" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057276 2531 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057280 2531 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057282 2531 flags.go:64] FLAG: --manifest-url="" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057285 2531 flags.go:64] FLAG: --manifest-url-header="" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057289 2531 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057291 2531 flags.go:64] FLAG: --max-open-files="1000000" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057296 2531 flags.go:64] FLAG: --max-pods="110" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057299 2531 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057301 2531 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057304 2531 flags.go:64] FLAG: --memory-manager-policy="None" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057307 2531 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057310 2531 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 24 14:56:08.064257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057312 2531 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057315 2531 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057322 2531 flags.go:64] FLAG: --node-status-max-images="50" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057325 2531 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057328 2531 flags.go:64] FLAG: --oom-score-adj="-999" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057331 2531 flags.go:64] FLAG: --pod-cidr="" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057333 2531 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057339 2531 flags.go:64] FLAG: --pod-manifest-path="" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057342 2531 flags.go:64] FLAG: --pod-max-pids="-1" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057346 2531 flags.go:64] FLAG: --pods-per-core="0" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057349 2531 flags.go:64] FLAG: --port="10250" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057352 2531 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057354 2531 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f21311577153a4e2" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057357 2531 flags.go:64] FLAG: --qos-reserved="" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057360 2531 flags.go:64] FLAG: --read-only-port="10255" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057363 2531 flags.go:64] FLAG: --register-node="true" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057366 2531 flags.go:64] FLAG: --register-with-taints="" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057369 2531 flags.go:64] FLAG: --registry-burst="10" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057373 2531 flags.go:64] FLAG: --registry-qps="5" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057375 2531 flags.go:64] FLAG: --reserved-cpus="" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057378 2531 flags.go:64] FLAG: --reserved-memory="" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057381 2531 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057384 2531 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057387 2531 flags.go:64] FLAG: --rotate-certificates="false" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057389 2531 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 24 14:56:08.064876 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057392 2531 flags.go:64] FLAG: --runonce="false" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057395 2531 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057398 2531 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057401 2531 flags.go:64] FLAG: --seccomp-default="false" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057403 2531 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057406 2531 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057409 2531 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057411 2531 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057414 2531 flags.go:64] FLAG: --storage-driver-password="root" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057417 2531 flags.go:64] FLAG: --storage-driver-secure="false" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057419 2531 flags.go:64] FLAG: --storage-driver-table="stats" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057422 2531 flags.go:64] FLAG: --storage-driver-user="root" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057424 2531 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057427 2531 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057430 2531 flags.go:64] FLAG: --system-cgroups="" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057432 2531 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057440 2531 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057442 2531 flags.go:64] FLAG: --tls-cert-file="" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057445 2531 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057452 2531 flags.go:64] FLAG: --tls-min-version="" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057454 2531 flags.go:64] FLAG: --tls-private-key-file="" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057457 2531 flags.go:64] FLAG: --topology-manager-policy="none" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057460 2531 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057462 2531 flags.go:64] FLAG: --topology-manager-scope="container" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057465 2531 flags.go:64] FLAG: --v="2" Jul 24 14:56:08.065476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057469 2531 flags.go:64] FLAG: --version="false" Jul 24 14:56:08.066087 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057474 2531 flags.go:64] FLAG: --vmodule="" Jul 24 14:56:08.066087 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057477 2531 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 24 14:56:08.066087 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.057480 2531 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 24 14:56:08.066087 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057588 2531 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 14:56:08.066087 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057592 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 14:56:08.066087 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057595 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 14:56:08.066087 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057598 2531 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 14:56:08.066087 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057600 2531 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 14:56:08.066087 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057603 2531 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 14:56:08.066087 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057605 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 14:56:08.066087 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057608 2531 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 14:56:08.066087 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057611 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 14:56:08.066087 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057613 2531 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 14:56:08.066087 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057616 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 14:56:08.066087 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057618 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 14:56:08.066087 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057621 2531 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 14:56:08.066087 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057624 2531 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 14:56:08.066087 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057628 2531 feature_gate.go:328] unrecognized feature gate: Example Jul 24 14:56:08.066087 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057631 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 14:56:08.066087 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057633 2531 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 14:56:08.066589 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057636 2531 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 14:56:08.066589 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057639 2531 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 14:56:08.066589 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057643 2531 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 14:56:08.066589 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057646 2531 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 14:56:08.066589 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057648 2531 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 14:56:08.066589 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057651 2531 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 14:56:08.066589 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057653 2531 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 14:56:08.066589 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057655 2531 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 14:56:08.066589 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057658 2531 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 14:56:08.066589 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057660 2531 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 14:56:08.066589 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057663 2531 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 14:56:08.066589 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057665 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 14:56:08.066589 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057668 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 14:56:08.066589 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057670 2531 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 14:56:08.066589 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057673 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 14:56:08.066589 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057676 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 14:56:08.066589 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057678 2531 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 14:56:08.066589 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057681 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 14:56:08.066589 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057683 2531 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 14:56:08.066589 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057685 2531 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 14:56:08.067225 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057690 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 14:56:08.067225 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057693 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 14:56:08.067225 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057696 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 14:56:08.067225 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057698 2531 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 14:56:08.067225 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057701 2531 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 14:56:08.067225 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057703 2531 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 14:56:08.067225 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057705 2531 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 14:56:08.067225 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057708 2531 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 14:56:08.067225 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057710 2531 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 14:56:08.067225 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057713 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 14:56:08.067225 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057715 2531 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 14:56:08.067225 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057718 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 14:56:08.067225 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057720 2531 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 14:56:08.067225 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057723 2531 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 14:56:08.067225 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057727 2531 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 14:56:08.067225 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057729 2531 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 14:56:08.067225 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057732 2531 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 14:56:08.067225 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057734 2531 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 14:56:08.067225 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057736 2531 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 14:56:08.067225 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057739 2531 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 14:56:08.067710 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057742 2531 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 14:56:08.067710 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057744 2531 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 14:56:08.067710 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057746 2531 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 14:56:08.067710 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057749 2531 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 14:56:08.067710 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057751 2531 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 14:56:08.067710 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057754 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 14:56:08.067710 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057757 2531 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 14:56:08.067710 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057760 2531 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 14:56:08.067710 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057762 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 14:56:08.067710 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057765 2531 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 14:56:08.067710 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057767 2531 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 14:56:08.067710 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057769 2531 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 14:56:08.067710 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057773 2531 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 14:56:08.067710 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057776 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 14:56:08.067710 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057778 2531 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 14:56:08.067710 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057781 2531 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 14:56:08.067710 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057784 2531 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 14:56:08.067710 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057786 2531 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 14:56:08.067710 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057788 2531 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 14:56:08.067710 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057791 2531 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 14:56:08.068216 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057793 2531 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 14:56:08.068216 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057796 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 14:56:08.068216 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057798 2531 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 14:56:08.068216 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057801 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 14:56:08.068216 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057803 2531 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 14:56:08.068216 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057806 2531 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 14:56:08.068216 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057812 2531 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 14:56:08.068216 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057815 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 14:56:08.068216 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057817 2531 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 14:56:08.068216 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057821 2531 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 14:56:08.068216 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057824 2531 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 14:56:08.068216 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057828 2531 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 14:56:08.068216 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057832 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 14:56:08.068216 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057835 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 14:56:08.068216 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057838 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 14:56:08.068216 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057840 2531 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 14:56:08.068216 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057843 2531 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 14:56:08.068216 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057846 2531 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 14:56:08.068216 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057848 2531 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 14:56:08.068681 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057851 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 14:56:08.068681 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057853 2531 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 14:56:08.068681 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.057856 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 14:56:08.068681 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.058452 2531 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 14:56:08.068681 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.065450 2531 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 24 14:56:08.068681 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.065464 2531 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 24 14:56:08.068681 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.065482 2531 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 24 14:56:08.068681 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.065488 2531 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 24 14:56:08.068681 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065544 2531 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 14:56:08.068681 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065549 2531 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 14:56:08.068681 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065552 2531 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 14:56:08.068681 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065555 2531 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 14:56:08.068681 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065558 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 14:56:08.068681 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065560 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 14:56:08.068681 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065563 2531 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 14:56:08.069075 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065565 2531 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 14:56:08.069075 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065568 2531 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 14:56:08.069075 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065571 2531 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 14:56:08.069075 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065573 2531 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 14:56:08.069075 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065576 2531 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 14:56:08.069075 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065579 2531 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 14:56:08.069075 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065581 2531 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 14:56:08.069075 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065584 2531 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 14:56:08.069075 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065587 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 14:56:08.069075 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065590 2531 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 14:56:08.069075 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065592 2531 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 14:56:08.069075 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065594 2531 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 14:56:08.069075 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065599 2531 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 14:56:08.069075 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065603 2531 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 14:56:08.069075 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065606 2531 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 14:56:08.069075 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065608 2531 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 14:56:08.069075 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065611 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 14:56:08.069075 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065615 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 14:56:08.069075 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065617 2531 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 14:56:08.069533 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065620 2531 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 14:56:08.069533 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065622 2531 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 14:56:08.069533 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065625 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 14:56:08.069533 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065628 2531 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 14:56:08.069533 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065631 2531 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 14:56:08.069533 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065633 2531 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 14:56:08.069533 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065636 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 14:56:08.069533 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065638 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 14:56:08.069533 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065641 2531 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 14:56:08.069533 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065643 2531 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 14:56:08.069533 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065646 2531 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 14:56:08.069533 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065648 2531 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 14:56:08.069533 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065650 2531 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 14:56:08.069533 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065654 2531 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 14:56:08.069533 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065658 2531 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 14:56:08.069533 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065660 2531 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 14:56:08.069533 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065663 2531 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 14:56:08.069533 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065666 2531 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 14:56:08.069533 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065669 2531 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 14:56:08.070011 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065672 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 14:56:08.070011 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065674 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 14:56:08.070011 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065677 2531 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 14:56:08.070011 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065680 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 14:56:08.070011 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065683 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 14:56:08.070011 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065685 2531 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 14:56:08.070011 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065688 2531 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 14:56:08.070011 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065690 2531 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 14:56:08.070011 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065693 2531 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 14:56:08.070011 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065696 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 14:56:08.070011 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065698 2531 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 14:56:08.070011 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065701 2531 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 14:56:08.070011 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065704 2531 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 14:56:08.070011 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065706 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 14:56:08.070011 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065709 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 14:56:08.070011 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065712 2531 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 14:56:08.070011 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065715 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 14:56:08.070011 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065718 2531 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 14:56:08.070011 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065720 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 14:56:08.070011 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065723 2531 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 14:56:08.070507 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065725 2531 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 14:56:08.070507 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065727 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 14:56:08.070507 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065730 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 14:56:08.070507 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065732 2531 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 14:56:08.070507 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065735 2531 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 14:56:08.070507 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065737 2531 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 14:56:08.070507 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065740 2531 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 14:56:08.070507 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065742 2531 feature_gate.go:328] unrecognized feature gate: Example Jul 24 14:56:08.070507 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065745 2531 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 14:56:08.070507 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065747 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 14:56:08.070507 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065750 2531 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 14:56:08.070507 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065752 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 14:56:08.070507 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065755 2531 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 14:56:08.070507 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065757 2531 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 14:56:08.070507 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065760 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 14:56:08.070507 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065762 2531 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 14:56:08.070507 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065764 2531 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 14:56:08.070507 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065767 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 14:56:08.070507 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065769 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 14:56:08.070984 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065773 2531 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 14:56:08.070984 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065776 2531 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 14:56:08.070984 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065779 2531 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 14:56:08.070984 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065782 2531 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 14:56:08.070984 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065785 2531 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 14:56:08.070984 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065788 2531 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 14:56:08.070984 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065791 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 14:56:08.070984 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065794 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 14:56:08.070984 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065797 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 14:56:08.070984 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065799 2531 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 14:56:08.070984 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065802 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 14:56:08.070984 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065804 2531 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 14:56:08.070984 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065807 2531 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 14:56:08.070984 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065809 2531 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 14:56:08.070984 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065812 2531 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 14:56:08.071339 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.065816 2531 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 14:56:08.071339 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065985 2531 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 14:56:08.071339 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065991 2531 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 14:56:08.071339 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065994 2531 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 14:56:08.071339 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.065996 2531 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 14:56:08.071339 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066000 2531 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 14:56:08.071339 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066002 2531 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 14:56:08.071339 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066005 2531 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 14:56:08.071339 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066008 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 14:56:08.071339 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066010 2531 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 14:56:08.071339 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066014 2531 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 14:56:08.071339 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066018 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 14:56:08.071339 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066020 2531 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 14:56:08.071339 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066023 2531 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 14:56:08.071675 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066026 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 14:56:08.071675 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066028 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 14:56:08.071675 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066031 2531 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 14:56:08.071675 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066033 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 14:56:08.071675 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066036 2531 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 14:56:08.071675 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066038 2531 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 14:56:08.071675 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066041 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 14:56:08.071675 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066043 2531 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 14:56:08.071675 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066045 2531 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 14:56:08.071675 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066048 2531 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 14:56:08.071675 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066051 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 14:56:08.071675 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066053 2531 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 14:56:08.071675 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066055 2531 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 14:56:08.071675 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066058 2531 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 14:56:08.071675 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066061 2531 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 14:56:08.071675 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066063 2531 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 14:56:08.071675 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066065 2531 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 14:56:08.071675 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066068 2531 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 14:56:08.071675 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066070 2531 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 14:56:08.071675 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066072 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 14:56:08.072248 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066075 2531 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 14:56:08.072248 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066077 2531 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 14:56:08.072248 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066080 2531 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 14:56:08.072248 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066082 2531 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 14:56:08.072248 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066084 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 14:56:08.072248 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066087 2531 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 14:56:08.072248 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066090 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 14:56:08.072248 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066092 2531 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 14:56:08.072248 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066095 2531 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 14:56:08.072248 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066097 2531 feature_gate.go:328] unrecognized feature gate: Example Jul 24 14:56:08.072248 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066100 2531 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 14:56:08.072248 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066103 2531 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 14:56:08.072248 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066105 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 14:56:08.072248 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066107 2531 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 14:56:08.072248 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066110 2531 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 14:56:08.072248 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066112 2531 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 14:56:08.072248 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066115 2531 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 14:56:08.072248 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066117 2531 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 14:56:08.072248 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066119 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 14:56:08.072248 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066122 2531 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 14:56:08.072759 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066124 2531 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 14:56:08.072759 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066126 2531 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 14:56:08.072759 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066129 2531 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 14:56:08.072759 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066132 2531 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 14:56:08.072759 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066134 2531 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 14:56:08.072759 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066137 2531 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 14:56:08.072759 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066140 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 14:56:08.072759 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066142 2531 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 14:56:08.072759 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066145 2531 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 14:56:08.072759 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066147 2531 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 14:56:08.072759 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066149 2531 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 14:56:08.072759 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066152 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 14:56:08.072759 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066154 2531 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 14:56:08.072759 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066157 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 14:56:08.072759 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066159 2531 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 14:56:08.072759 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066161 2531 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 14:56:08.072759 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066164 2531 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 14:56:08.072759 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066166 2531 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 14:56:08.072759 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066169 2531 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 14:56:08.072759 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066171 2531 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 14:56:08.073256 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066174 2531 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 14:56:08.073256 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066176 2531 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 14:56:08.073256 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066179 2531 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 14:56:08.073256 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066181 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 14:56:08.073256 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066183 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 14:56:08.073256 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066186 2531 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 14:56:08.073256 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066188 2531 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 14:56:08.073256 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066190 2531 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 14:56:08.073256 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066193 2531 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 14:56:08.073256 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066195 2531 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 14:56:08.073256 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066197 2531 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 14:56:08.073256 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066199 2531 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 14:56:08.073256 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066203 2531 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 14:56:08.073256 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066205 2531 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 14:56:08.073256 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066208 2531 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 14:56:08.073256 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066211 2531 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 14:56:08.073256 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066214 2531 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 14:56:08.073256 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066216 2531 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 14:56:08.073256 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066219 2531 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 14:56:08.073724 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066222 2531 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 14:56:08.073724 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066224 2531 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 14:56:08.073724 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066227 2531 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 14:56:08.073724 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066229 2531 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 14:56:08.073724 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066232 2531 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 14:56:08.073724 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066234 2531 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 14:56:08.073724 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:08.066238 2531 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 14:56:08.073724 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.066243 2531 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 14:56:08.073724 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.066340 2531 server.go:962] "Client rotation is on, will bootstrap in background" Jul 24 14:56:08.073724 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.068469 2531 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 24 14:56:08.073724 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.069898 2531 server.go:1019] "Starting client certificate rotation" Jul 24 14:56:08.073724 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.070001 2531 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 14:56:08.073724 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.070035 2531 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 14:56:08.074072 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.073347 2531 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 14:56:08.079059 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.079046 2531 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 14:56:08.089953 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.089938 2531 log.go:25] "Validated CRI v1 runtime API" Jul 24 14:56:08.091844 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.091826 2531 log.go:25] "Validated CRI v1 image API" Jul 24 14:56:08.093250 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.093225 2531 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 24 14:56:08.095697 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.095679 2531 fs.go:135] Filesystem UUIDs: map[4f17c3a7-45e9-47d9-9a77-8041767c8eb2:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 7d8262d4-ed47-48ff-9520-a9d32a0b3ab0:/dev/nvme0n1p4] Jul 24 14:56:08.095760 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.095696 2531 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 24 14:56:08.101583 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.101480 2531 manager.go:216] Machine: {Timestamp:2026-07-24 14:56:08.099425076 +0000 UTC m=+0.329901761 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100103 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2149b9a4909ba38b4ef2c4eb44162e SystemUUID:ec2149b9-a490-9ba3-8b4e-f2c4eb44162e BootID:13e31eef-3f74-4dcf-9476-9f8be0bd965e Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:01:6a:2d:ed:ed Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:01:6a:2d:ed:ed Speed:0 Mtu:9001} {Name:ovs-system MacAddress:22:c4:a8:7d:5a:ea Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 24 14:56:08.101583 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.101580 2531 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 24 14:56:08.101689 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.101680 2531 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 24 14:56:08.102703 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.102682 2531 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 24 14:56:08.102823 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.102705 2531 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-108.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 24 14:56:08.103430 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.103421 2531 topology_manager.go:138] "Creating topology manager with none policy" Jul 24 14:56:08.103468 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.103433 2531 container_manager_linux.go:309] "Creating device plugin manager" Jul 24 14:56:08.103468 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.103440 2531 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 14:56:08.104249 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.104240 2531 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 14:56:08.105031 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.105021 2531 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 24 14:56:08.106819 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.106805 2531 state_mem.go:36] "Initialized new in-memory state store" Jul 24 14:56:08.106971 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.106962 2531 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 24 14:56:08.107692 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.107683 2531 kubelet.go:486] "Attempting to sync node with API server" Jul 24 14:56:08.107748 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.107702 2531 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 24 14:56:08.107748 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.107720 2531 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 24 14:56:08.107748 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.107728 2531 kubelet.go:398] "Adding apiserver pod source" Jul 24 14:56:08.107748 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.107737 2531 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 24 14:56:08.108104 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.108093 2531 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 14:56:08.108136 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.108111 2531 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 14:56:08.110444 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.110423 2531 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 24 14:56:08.112368 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.112355 2531 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 24 14:56:08.112427 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.112385 2531 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 24 14:56:08.112623 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.112610 2531 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 14:56:08.114069 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.114044 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 24 14:56:08.114150 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.114081 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 24 14:56:08.114150 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.114096 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 24 14:56:08.114150 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.114105 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 24 14:56:08.114150 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.114119 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 24 14:56:08.114150 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.114127 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 24 14:56:08.114150 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.114150 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 24 14:56:08.114308 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.114158 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 24 14:56:08.114308 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.114173 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 24 14:56:08.114308 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.114182 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 24 14:56:08.114308 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.114194 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 24 14:56:08.114308 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.114202 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 24 14:56:08.114812 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.114800 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 24 14:56:08.114847 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.114813 2531 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 24 14:56:08.118018 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.118005 2531 server.go:1268] "Started kubelet" Jul 24 14:56:08.118397 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.118381 2531 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 24 14:56:08.118644 ip-10-0-137-108 systemd[1]: Started Kubernetes Kubelet. Jul 24 14:56:08.118952 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.118819 2531 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 24 14:56:08.118952 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.118950 2531 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 24 14:56:08.119954 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.119920 2531 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-108.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 24 14:56:08.120057 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.119980 2531 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 24 14:56:08.122429 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.122407 2531 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-108.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 24 14:56:08.122850 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.122812 2531 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 24 14:56:08.124007 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.123978 2531 server.go:310] "Adding debug handlers to kubelet server" Jul 24 14:56:08.124146 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.124122 2531 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 24 14:56:08.127530 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.126540 2531 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-108.ec2.internal.18c5419c9680a1ad default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-108.ec2.internal,UID:ip-10-0-137-108.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-108.ec2.internal,},FirstTimestamp:2026-07-24 14:56:08.117985709 +0000 UTC m=+0.348462384,LastTimestamp:2026-07-24 14:56:08.117985709 +0000 UTC m=+0.348462384,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-108.ec2.internal,}" Jul 24 14:56:08.128959 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.128940 2531 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 24 14:56:08.129042 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.128945 2531 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 24 14:56:08.129095 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.129043 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-108.ec2.internal\" not found" Jul 24 14:56:08.129901 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.129882 2531 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 24 14:56:08.129901 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.129890 2531 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 24 14:56:08.130027 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.129913 2531 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 24 14:56:08.130027 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.130017 2531 reconstruct.go:97] "Volume reconstruction finished" Jul 24 14:56:08.130027 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.130023 2531 reconciler.go:29] "Reconciler: start to sync state" Jul 24 14:56:08.131162 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.131145 2531 factory.go:55] Registering systemd factory Jul 24 14:56:08.131220 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.131167 2531 factory.go:223] Registration of the systemd container factory successfully Jul 24 14:56:08.131364 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.131354 2531 factory.go:153] Registering CRI-O factory Jul 24 14:56:08.131423 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.131365 2531 factory.go:223] Registration of the crio container factory successfully Jul 24 14:56:08.131459 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.131440 2531 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 24 14:56:08.131488 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.131473 2531 factory.go:103] Registering Raw factory Jul 24 14:56:08.131488 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.131485 2531 manager.go:1196] Started watching for new ooms in manager Jul 24 14:56:08.131906 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.131895 2531 manager.go:349] Starting recovery of all containers Jul 24 14:56:08.139896 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.139850 2531 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-108.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 24 14:56:08.139994 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.139897 2531 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 24 14:56:08.143069 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.143007 2531 manager.go:354] Recovery completed Jul 24 14:56:08.147318 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.147307 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 14:56:08.149320 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.149291 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientMemory" Jul 24 14:56:08.149397 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.149335 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 14:56:08.149397 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.149350 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientPID" Jul 24 14:56:08.149908 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.149895 2531 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 24 14:56:08.149908 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.149896 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 14:56:08.149995 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.149910 2531 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 24 14:56:08.149995 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.149960 2531 state_mem.go:36] "Initialized new in-memory state store" Jul 24 14:56:08.150833 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.150821 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientMemory" Jul 24 14:56:08.150907 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.150844 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 14:56:08.150907 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.150858 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientPID" Jul 24 14:56:08.151899 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.151847 2531 policy_none.go:49] "None policy: Start" Jul 24 14:56:08.151899 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.151890 2531 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 24 14:56:08.151899 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.151902 2531 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 24 14:56:08.152889 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.152858 2531 policy_none.go:47] "Start" Jul 24 14:56:08.160270 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.160186 2531 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-108.ec2.internal.18c5419c985ec293 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-108.ec2.internal,UID:ip-10-0-137-108.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-108.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-108.ec2.internal,},FirstTimestamp:2026-07-24 14:56:08.149320339 +0000 UTC m=+0.379797029,LastTimestamp:2026-07-24 14:56:08.149320339 +0000 UTC m=+0.379797029,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-108.ec2.internal,}" Jul 24 14:56:08.171333 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.171248 2531 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-108.ec2.internal.18c5419c985f1700 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-108.ec2.internal,UID:ip-10-0-137-108.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-108.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-108.ec2.internal,},FirstTimestamp:2026-07-24 14:56:08.149341952 +0000 UTC m=+0.379818823,LastTimestamp:2026-07-24 14:56:08.149341952 +0000 UTC m=+0.379818823,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-108.ec2.internal,}" Jul 24 14:56:08.172492 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.172465 2531 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dtqdv" Jul 24 14:56:08.182146 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.182030 2531 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dtqdv" Jul 24 14:56:08.187415 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.187357 2531 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-108.ec2.internal.18c5419c985f4702 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-108.ec2.internal,UID:ip-10-0-137-108.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-137-108.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-137-108.ec2.internal,},FirstTimestamp:2026-07-24 14:56:08.149354242 +0000 UTC m=+0.379830918,LastTimestamp:2026-07-24 14:56:08.149354242 +0000 UTC m=+0.379830918,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-108.ec2.internal,}" Jul 24 14:56:08.189441 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.189428 2531 manager.go:337] "Starting Device Plugin manager" Jul 24 14:56:08.189497 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.189456 2531 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 24 14:56:08.189497 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.189465 2531 server.go:87] "Starting device plugin registration server" Jul 24 14:56:08.189727 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.189713 2531 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 24 14:56:08.189812 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.189727 2531 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 24 14:56:08.189882 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.189811 2531 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 24 14:56:08.189945 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.189932 2531 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 24 14:56:08.189984 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.189946 2531 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 24 14:56:08.190448 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.190430 2531 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 24 14:56:08.190519 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.190489 2531 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-108.ec2.internal\" not found" Jul 24 14:56:08.269077 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.269047 2531 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 24 14:56:08.270361 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.270345 2531 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 24 14:56:08.270445 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.270382 2531 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 24 14:56:08.270445 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.270409 2531 kubelet.go:2448] "Starting kubelet main sync loop" Jul 24 14:56:08.270571 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.270448 2531 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 24 14:56:08.273300 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.273280 2531 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 14:56:08.290303 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.290280 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 14:56:08.291097 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.291077 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientMemory" Jul 24 14:56:08.291196 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.291113 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 14:56:08.291196 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.291128 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientPID" Jul 24 14:56:08.291196 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.291158 2531 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-108.ec2.internal" Jul 24 14:56:08.303851 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.303797 2531 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-108.ec2.internal" Jul 24 14:56:08.303851 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.303822 2531 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-108.ec2.internal\": node \"ip-10-0-137-108.ec2.internal\" not found" Jul 24 14:56:08.341915 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.341892 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-108.ec2.internal\" not found" Jul 24 14:56:08.370696 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.370660 2531 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal"] Jul 24 14:56:08.370782 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.370747 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 14:56:08.371587 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.371571 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientMemory" Jul 24 14:56:08.371648 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.371602 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 14:56:08.371648 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.371615 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientPID" Jul 24 14:56:08.372806 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.372794 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 14:56:08.372982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.372968 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" Jul 24 14:56:08.373027 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.372999 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 14:56:08.373533 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.373515 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientMemory" Jul 24 14:56:08.373589 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.373523 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientMemory" Jul 24 14:56:08.373589 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.373548 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 14:56:08.373589 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.373568 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 14:56:08.373589 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.373578 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientPID" Jul 24 14:56:08.373589 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.373584 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientPID" Jul 24 14:56:08.374899 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.374885 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal" Jul 24 14:56:08.374950 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.374912 2531 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 14:56:08.375537 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.375521 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientMemory" Jul 24 14:56:08.375603 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.375549 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 14:56:08.375603 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.375564 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientPID" Jul 24 14:56:08.389473 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.389450 2531 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-108.ec2.internal\" not found" node="ip-10-0-137-108.ec2.internal" Jul 24 14:56:08.393221 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.393206 2531 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-108.ec2.internal\" not found" node="ip-10-0-137-108.ec2.internal" Jul 24 14:56:08.431838 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.431817 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d99284ebfc3675b576b1ca8cc30be861-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal\" (UID: \"d99284ebfc3675b576b1ca8cc30be861\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" Jul 24 14:56:08.431941 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.431839 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d99284ebfc3675b576b1ca8cc30be861-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal\" (UID: \"d99284ebfc3675b576b1ca8cc30be861\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" Jul 24 14:56:08.431941 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.431856 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd97724e3608d55c7f74ae763e45fb90-config\") pod \"kube-apiserver-proxy-ip-10-0-137-108.ec2.internal\" (UID: \"fd97724e3608d55c7f74ae763e45fb90\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal" Jul 24 14:56:08.442927 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.442903 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-108.ec2.internal\" not found" Jul 24 14:56:08.532144 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.532107 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d99284ebfc3675b576b1ca8cc30be861-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal\" (UID: \"d99284ebfc3675b576b1ca8cc30be861\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" Jul 24 14:56:08.532144 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.532147 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d99284ebfc3675b576b1ca8cc30be861-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal\" (UID: \"d99284ebfc3675b576b1ca8cc30be861\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" Jul 24 14:56:08.532347 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.532202 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd97724e3608d55c7f74ae763e45fb90-config\") pod \"kube-apiserver-proxy-ip-10-0-137-108.ec2.internal\" (UID: \"fd97724e3608d55c7f74ae763e45fb90\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal" Jul 24 14:56:08.532347 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.532225 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d99284ebfc3675b576b1ca8cc30be861-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal\" (UID: \"d99284ebfc3675b576b1ca8cc30be861\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" Jul 24 14:56:08.532347 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.532247 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd97724e3608d55c7f74ae763e45fb90-config\") pod \"kube-apiserver-proxy-ip-10-0-137-108.ec2.internal\" (UID: \"fd97724e3608d55c7f74ae763e45fb90\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal" Jul 24 14:56:08.532347 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.532236 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d99284ebfc3675b576b1ca8cc30be861-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal\" (UID: \"d99284ebfc3675b576b1ca8cc30be861\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" Jul 24 14:56:08.543225 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.543201 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-108.ec2.internal\" not found" Jul 24 14:56:08.644078 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.644003 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-108.ec2.internal\" not found" Jul 24 14:56:08.694183 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.694159 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" Jul 24 14:56:08.698392 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:08.698105 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal" Jul 24 14:56:08.744278 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.744255 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-108.ec2.internal\" not found" Jul 24 14:56:08.844827 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.844791 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-108.ec2.internal\" not found" Jul 24 14:56:08.945315 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:08.945251 2531 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-108.ec2.internal\" not found" Jul 24 14:56:09.013211 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.013183 2531 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 14:56:09.030051 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.030032 2531 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal" Jul 24 14:56:09.039712 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.039688 2531 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 14:56:09.041536 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.041519 2531 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" Jul 24 14:56:09.053633 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.053606 2531 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 14:56:09.069767 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.069746 2531 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 24 14:56:09.069910 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.069893 2531 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 14:56:09.069968 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.069910 2531 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 14:56:09.108062 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.108044 2531 apiserver.go:52] "Watching apiserver" Jul 24 14:56:09.123314 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.123288 2531 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 24 14:56:09.124386 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.124363 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-6jf8n","kube-system/konnectivity-agent-fcmhs","openshift-dns/node-resolver-hcxz7","openshift-multus/multus-ff2nx","openshift-multus/network-metrics-daemon-xgswn","openshift-network-operator/iptables-alerter-s4h5d","kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq","openshift-cluster-node-tuning-operator/tuned-62nf6","openshift-image-registry/node-ca-pbnwh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal","openshift-multus/multus-additional-cni-plugins-vgqjg","openshift-network-diagnostics/network-check-target-mjqx9","openshift-ovn-kubernetes/ovnkube-node-f74fd"] Jul 24 14:56:09.125956 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.125938 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pbnwh" Jul 24 14:56:09.127033 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.127017 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:09.127098 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.127074 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mjqx9" podUID="ec175d02-f2a5-4492-8133-e642a367da32" Jul 24 14:56:09.128131 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.128113 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hcxz7" Jul 24 14:56:09.128417 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.128402 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 24 14:56:09.128495 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.128456 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 24 14:56:09.128495 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.128465 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 24 14:56:09.128596 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.128544 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mns5z\"" Jul 24 14:56:09.129082 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.129069 2531 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 24 14:56:09.130458 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.130435 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.131840 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.131815 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 24 14:56:09.131949 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.131924 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 24 14:56:09.132012 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.131937 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-27c2d\"" Jul 24 14:56:09.132285 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.132267 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:09.132343 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.132326 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 24 14:56:09.132343 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.132328 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xgswn" podUID="a9494b1d-f2bb-408b-8283-7d622ea5e450" Jul 24 14:56:09.132441 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.132424 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rt8j5\"" Jul 24 14:56:09.132914 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.132896 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 24 14:56:09.133021 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.132982 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 24 14:56:09.133083 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.133048 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 24 14:56:09.133896 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.133881 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s4h5d" Jul 24 14:56:09.134965 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.134951 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.135277 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135246 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-os-release\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.135277 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135276 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-host-run-netns\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.135406 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135291 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-hostroot\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.135406 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135306 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzjcr\" (UniqueName: \"kubernetes.io/projected/96ce8f23-8457-4b01-bde6-4caf871479b2-kube-api-access-fzjcr\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.135406 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135321 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4rz7\" (UniqueName: \"kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7\") pod \"network-check-target-mjqx9\" (UID: \"ec175d02-f2a5-4492-8133-e642a367da32\") " pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:09.135406 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135335 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdaadab4-2a42-46d7-8ca6-24881a9d54d0-tmp-dir\") pod \"node-resolver-hcxz7\" (UID: \"bdaadab4-2a42-46d7-8ca6-24881a9d54d0\") " pod="openshift-dns/node-resolver-hcxz7" Jul 24 14:56:09.135406 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135348 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prhh6\" (UniqueName: \"kubernetes.io/projected/bdaadab4-2a42-46d7-8ca6-24881a9d54d0-kube-api-access-prhh6\") pod \"node-resolver-hcxz7\" (UID: \"bdaadab4-2a42-46d7-8ca6-24881a9d54d0\") " pod="openshift-dns/node-resolver-hcxz7" Jul 24 14:56:09.135406 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135381 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-system-cni-dir\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.135632 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135409 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96ce8f23-8457-4b01-bde6-4caf871479b2-cni-binary-copy\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.135632 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135442 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96ce8f23-8457-4b01-bde6-4caf871479b2-multus-daemon-config\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.135632 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135463 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-host-run-multus-certs\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.135632 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135479 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91255a26-e33b-408d-81d1-b8ec24d1e7d6-serviceca\") pod \"node-ca-pbnwh\" (UID: \"91255a26-e33b-408d-81d1-b8ec24d1e7d6\") " pod="openshift-image-registry/node-ca-pbnwh" Jul 24 14:56:09.135632 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135504 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-multus-cni-dir\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.135632 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135532 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-host-run-k8s-cni-cncf-io\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.135632 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135565 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-multus-conf-dir\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.135632 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135581 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-host-var-lib-cni-multus\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.135632 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135596 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs\") pod \"network-metrics-daemon-xgswn\" (UID: \"a9494b1d-f2bb-408b-8283-7d622ea5e450\") " pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:09.135632 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135618 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91255a26-e33b-408d-81d1-b8ec24d1e7d6-host\") pod \"node-ca-pbnwh\" (UID: \"91255a26-e33b-408d-81d1-b8ec24d1e7d6\") " pod="openshift-image-registry/node-ca-pbnwh" Jul 24 14:56:09.136007 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135645 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zqf5\" (UniqueName: \"kubernetes.io/projected/a9494b1d-f2bb-408b-8283-7d622ea5e450-kube-api-access-5zqf5\") pod \"network-metrics-daemon-xgswn\" (UID: \"a9494b1d-f2bb-408b-8283-7d622ea5e450\") " pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:09.136007 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135672 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-host-var-lib-cni-bin\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.136007 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135723 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-etc-kubernetes\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.136007 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135775 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxcst\" (UniqueName: \"kubernetes.io/projected/91255a26-e33b-408d-81d1-b8ec24d1e7d6-kube-api-access-cxcst\") pod \"node-ca-pbnwh\" (UID: \"91255a26-e33b-408d-81d1-b8ec24d1e7d6\") " pod="openshift-image-registry/node-ca-pbnwh" Jul 24 14:56:09.136007 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135800 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 24 14:56:09.136007 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135804 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bdaadab4-2a42-46d7-8ca6-24881a9d54d0-hosts-file\") pod \"node-resolver-hcxz7\" (UID: \"bdaadab4-2a42-46d7-8ca6-24881a9d54d0\") " pod="openshift-dns/node-resolver-hcxz7" Jul 24 14:56:09.136007 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135809 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 24 14:56:09.136007 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135823 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-cnibin\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.136007 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135847 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-multus-socket-dir-parent\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.136007 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.135888 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-host-var-lib-kubelet\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.136386 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.136369 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 24 14:56:09.136445 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.136398 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.137183 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.137156 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 24 14:56:09.137183 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.137166 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 24 14:56:09.137316 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.137191 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wffbf\"" Jul 24 14:56:09.137316 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.137279 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ml8nh\"" Jul 24 14:56:09.137613 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.137600 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fcmhs" Jul 24 14:56:09.139142 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.139125 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:09.139223 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.139194 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jf8n" podUID="441a2a3d-5df6-4a88-92cc-63d18ba4310f" Jul 24 14:56:09.139666 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.139651 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 24 14:56:09.139759 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.139735 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 24 14:56:09.139888 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.139759 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 24 14:56:09.140116 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.140091 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 24 14:56:09.140116 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.140103 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 24 14:56:09.140781 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.140563 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.140781 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.140693 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-d85jv\"" Jul 24 14:56:09.140781 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.140728 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 24 14:56:09.141078 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.141059 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 24 14:56:09.141138 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.141125 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 24 14:56:09.141188 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.141125 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ptph8\"" Jul 24 14:56:09.142031 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.142018 2531 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 14:56:09.142143 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.142129 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.142622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.142607 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 24 14:56:09.142820 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.142807 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 24 14:56:09.143099 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.143083 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 24 14:56:09.143181 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.143109 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4l2hg\"" Jul 24 14:56:09.144253 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.144240 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 24 14:56:09.144634 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.144620 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-d6qhj\"" Jul 24 14:56:09.144787 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.144775 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 24 14:56:09.169039 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.169015 2531 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9wzg6" Jul 24 14:56:09.174911 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.174894 2531 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9wzg6" Jul 24 14:56:09.186190 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.186167 2531 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 14:51:09 +0000 UTC" deadline="2028-02-17 18:36:50.785772799 +0000 UTC" Jul 24 14:56:09.186190 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.186189 2531 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13755h40m41.599586496s" Jul 24 14:56:09.230636 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.230619 2531 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 24 14:56:09.236349 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236333 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8zzm\" (UniqueName: \"kubernetes.io/projected/c4c7fe7d-701d-40f5-aafb-7608ab866182-kube-api-access-w8zzm\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.236401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236364 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02657d23-d776-4da0-939a-e1f8893c5904-cnibin\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.236401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236383 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-var-lib-kubelet\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.236401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236397 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-etc-openvswitch\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.236501 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236449 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-os-release\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.236501 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236481 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-hostroot\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.236558 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236519 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-os-release\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.236558 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236549 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-cni-bin\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.236617 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236575 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-hostroot\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.236617 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236605 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4rz7\" (UniqueName: \"kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7\") pod \"network-check-target-mjqx9\" (UID: \"ec175d02-f2a5-4492-8133-e642a367da32\") " pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:09.236684 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236625 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdaadab4-2a42-46d7-8ca6-24881a9d54d0-tmp-dir\") pod \"node-resolver-hcxz7\" (UID: \"bdaadab4-2a42-46d7-8ca6-24881a9d54d0\") " pod="openshift-dns/node-resolver-hcxz7" Jul 24 14:56:09.236684 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236654 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-system-cni-dir\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.236684 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236676 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96ce8f23-8457-4b01-bde6-4caf871479b2-multus-daemon-config\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.236809 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236700 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-etc-sysctl-d\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.236809 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236723 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-etc-systemd\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.236809 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236745 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/441a2a3d-5df6-4a88-92cc-63d18ba4310f-kubelet-config\") pod \"global-pull-secret-syncer-6jf8n\" (UID: \"441a2a3d-5df6-4a88-92cc-63d18ba4310f\") " pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:09.236809 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236767 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/447552f9-3427-4e02-9ad1-fa205e53644f-ovnkube-config\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.236809 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236777 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-system-cni-dir\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.236809 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236790 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91255a26-e33b-408d-81d1-b8ec24d1e7d6-serviceca\") pod \"node-ca-pbnwh\" (UID: \"91255a26-e33b-408d-81d1-b8ec24d1e7d6\") " pod="openshift-image-registry/node-ca-pbnwh" Jul 24 14:56:09.237024 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236845 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-host-run-k8s-cni-cncf-io\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.237024 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236888 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdaadab4-2a42-46d7-8ca6-24881a9d54d0-tmp-dir\") pod \"node-resolver-hcxz7\" (UID: \"bdaadab4-2a42-46d7-8ca6-24881a9d54d0\") " pod="openshift-dns/node-resolver-hcxz7" Jul 24 14:56:09.237024 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236897 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-sys\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.237024 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236923 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-lib-modules\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.237024 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236924 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-host-run-k8s-cni-cncf-io\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.237024 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236947 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a82a6065-127d-4ced-9487-59b5c2849dbc-agent-certs\") pod \"konnectivity-agent-fcmhs\" (UID: \"a82a6065-127d-4ced-9487-59b5c2849dbc\") " pod="kube-system/konnectivity-agent-fcmhs" Jul 24 14:56:09.237024 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.236976 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-systemd-units\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.237024 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237004 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-var-lib-openvswitch\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.237307 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237028 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4c7fe7d-701d-40f5-aafb-7608ab866182-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.237307 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237050 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-multus-conf-dir\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.237307 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237096 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-multus-conf-dir\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.237307 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237075 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02657d23-d776-4da0-939a-e1f8893c5904-cni-binary-copy\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.237307 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237130 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-etc-kubernetes\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.237307 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237147 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-host-var-lib-cni-multus\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.237307 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237162 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs\") pod \"network-metrics-daemon-xgswn\" (UID: \"a9494b1d-f2bb-408b-8283-7d622ea5e450\") " pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:09.237307 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237184 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02657d23-d776-4da0-939a-e1f8893c5904-system-cni-dir\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.237307 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237205 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-run\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.237307 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237236 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/96ce8f23-8457-4b01-bde6-4caf871479b2-multus-daemon-config\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.237307 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237244 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fde578a4-ae0f-425f-991c-30917584906f-etc-tuned\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.237307 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237252 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-host-var-lib-cni-multus\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.237307 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237287 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/441a2a3d-5df6-4a88-92cc-63d18ba4310f-dbus\") pod \"global-pull-secret-syncer-6jf8n\" (UID: \"441a2a3d-5df6-4a88-92cc-63d18ba4310f\") " pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:09.237307 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.237296 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 14:56:09.238041 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237325 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-run-netns\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.238041 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237360 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-run-ovn\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.238041 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237381 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91255a26-e33b-408d-81d1-b8ec24d1e7d6-host\") pod \"node-ca-pbnwh\" (UID: \"91255a26-e33b-408d-81d1-b8ec24d1e7d6\") " pod="openshift-image-registry/node-ca-pbnwh" Jul 24 14:56:09.238041 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.237411 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs podName:a9494b1d-f2bb-408b-8283-7d622ea5e450 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:09.737388836 +0000 UTC m=+1.967865521 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs") pod "network-metrics-daemon-xgswn" (UID: "a9494b1d-f2bb-408b-8283-7d622ea5e450") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 14:56:09.238041 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237416 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91255a26-e33b-408d-81d1-b8ec24d1e7d6-host\") pod \"node-ca-pbnwh\" (UID: \"91255a26-e33b-408d-81d1-b8ec24d1e7d6\") " pod="openshift-image-registry/node-ca-pbnwh" Jul 24 14:56:09.238041 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237441 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zqf5\" (UniqueName: \"kubernetes.io/projected/a9494b1d-f2bb-408b-8283-7d622ea5e450-kube-api-access-5zqf5\") pod \"network-metrics-daemon-xgswn\" (UID: \"a9494b1d-f2bb-408b-8283-7d622ea5e450\") " pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:09.238041 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237465 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-run-openvswitch\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.238041 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237492 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4c7fe7d-701d-40f5-aafb-7608ab866182-etc-selinux\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.238041 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237515 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a82a6065-127d-4ced-9487-59b5c2849dbc-konnectivity-ca\") pod \"konnectivity-agent-fcmhs\" (UID: \"a82a6065-127d-4ced-9487-59b5c2849dbc\") " pod="kube-system/konnectivity-agent-fcmhs" Jul 24 14:56:09.238041 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237541 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-host-var-lib-cni-bin\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.238041 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237564 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bdaadab4-2a42-46d7-8ca6-24881a9d54d0-hosts-file\") pod \"node-resolver-hcxz7\" (UID: \"bdaadab4-2a42-46d7-8ca6-24881a9d54d0\") " pod="openshift-dns/node-resolver-hcxz7" Jul 24 14:56:09.238041 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237586 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-cnibin\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.238041 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237589 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-host-var-lib-cni-bin\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.238041 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237640 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-cnibin\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.238041 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237657 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bdaadab4-2a42-46d7-8ca6-24881a9d54d0-hosts-file\") pod \"node-resolver-hcxz7\" (UID: \"bdaadab4-2a42-46d7-8ca6-24881a9d54d0\") " pod="openshift-dns/node-resolver-hcxz7" Jul 24 14:56:09.238041 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237684 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-multus-socket-dir-parent\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.238041 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237720 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fde578a4-ae0f-425f-991c-30917584906f-tmp\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.238607 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237740 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-host-var-lib-kubelet\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.238607 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237765 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/02657d23-d776-4da0-939a-e1f8893c5904-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.238607 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237783 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrr9b\" (UniqueName: \"kubernetes.io/projected/02657d23-d776-4da0-939a-e1f8893c5904-kube-api-access-qrr9b\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.238607 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237798 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-host\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.238607 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237812 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-host-var-lib-kubelet\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.238607 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237814 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91255a26-e33b-408d-81d1-b8ec24d1e7d6-serviceca\") pod \"node-ca-pbnwh\" (UID: \"91255a26-e33b-408d-81d1-b8ec24d1e7d6\") " pod="openshift-image-registry/node-ca-pbnwh" Jul 24 14:56:09.238607 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237836 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-run-ovn-kubernetes\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.238607 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237856 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-multus-socket-dir-parent\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.238607 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237857 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4c7fe7d-701d-40f5-aafb-7608ab866182-sys-fs\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.238607 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237902 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-host-run-netns\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.238607 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237917 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzjcr\" (UniqueName: \"kubernetes.io/projected/96ce8f23-8457-4b01-bde6-4caf871479b2-kube-api-access-fzjcr\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.238607 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237934 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9c1cadc9-4ba8-451f-b647-22451763a4cc-iptables-alerter-script\") pod \"iptables-alerter-s4h5d\" (UID: \"9c1cadc9-4ba8-451f-b647-22451763a4cc\") " pod="openshift-network-operator/iptables-alerter-s4h5d" Jul 24 14:56:09.238607 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237944 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-host-run-netns\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.238607 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237951 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-slash\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.238607 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.237976 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-log-socket\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.238607 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238008 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prhh6\" (UniqueName: \"kubernetes.io/projected/bdaadab4-2a42-46d7-8ca6-24881a9d54d0-kube-api-access-prhh6\") pod \"node-resolver-hcxz7\" (UID: \"bdaadab4-2a42-46d7-8ca6-24881a9d54d0\") " pod="openshift-dns/node-resolver-hcxz7" Jul 24 14:56:09.238607 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238032 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96ce8f23-8457-4b01-bde6-4caf871479b2-cni-binary-copy\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.239093 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238061 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-host-run-multus-certs\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.239093 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238083 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/02657d23-d776-4da0-939a-e1f8893c5904-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.239093 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238136 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-host-run-multus-certs\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.239093 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238154 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-etc-modprobe-d\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.239093 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238180 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-etc-sysconfig\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.239093 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238198 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m72hb\" (UniqueName: \"kubernetes.io/projected/fde578a4-ae0f-425f-991c-30917584906f-kube-api-access-m72hb\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.239093 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238218 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c1cadc9-4ba8-451f-b647-22451763a4cc-host-slash\") pod \"iptables-alerter-s4h5d\" (UID: \"9c1cadc9-4ba8-451f-b647-22451763a4cc\") " pod="openshift-network-operator/iptables-alerter-s4h5d" Jul 24 14:56:09.239093 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238242 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-multus-cni-dir\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.239093 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238256 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-etc-sysctl-conf\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.239093 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238271 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/447552f9-3427-4e02-9ad1-fa205e53644f-ovnkube-script-lib\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.239093 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238286 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-multus-cni-dir\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.239093 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238286 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8w4v\" (UniqueName: \"kubernetes.io/projected/447552f9-3427-4e02-9ad1-fa205e53644f-kube-api-access-p8w4v\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.239093 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238318 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02657d23-d776-4da0-939a-e1f8893c5904-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.239093 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238341 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-run-systemd\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.239093 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238356 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-node-log\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.239093 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238370 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/447552f9-3427-4e02-9ad1-fa205e53644f-env-overrides\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.239093 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238418 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/447552f9-3427-4e02-9ad1-fa205e53644f-ovn-node-metrics-cert\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.239544 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238427 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96ce8f23-8457-4b01-bde6-4caf871479b2-cni-binary-copy\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.239544 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238437 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02657d23-d776-4da0-939a-e1f8893c5904-os-release\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.239544 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238464 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret\") pod \"global-pull-secret-syncer-6jf8n\" (UID: \"441a2a3d-5df6-4a88-92cc-63d18ba4310f\") " pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:09.239544 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238495 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6gzs\" (UniqueName: \"kubernetes.io/projected/9c1cadc9-4ba8-451f-b647-22451763a4cc-kube-api-access-b6gzs\") pod \"iptables-alerter-s4h5d\" (UID: \"9c1cadc9-4ba8-451f-b647-22451763a4cc\") " pod="openshift-network-operator/iptables-alerter-s4h5d" Jul 24 14:56:09.239544 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238512 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.239544 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238526 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4c7fe7d-701d-40f5-aafb-7608ab866182-socket-dir\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.239544 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238540 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4c7fe7d-701d-40f5-aafb-7608ab866182-registration-dir\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.239544 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238553 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4c7fe7d-701d-40f5-aafb-7608ab866182-device-dir\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.239544 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238579 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-etc-kubernetes\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.239544 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238603 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxcst\" (UniqueName: \"kubernetes.io/projected/91255a26-e33b-408d-81d1-b8ec24d1e7d6-kube-api-access-cxcst\") pod \"node-ca-pbnwh\" (UID: \"91255a26-e33b-408d-81d1-b8ec24d1e7d6\") " pod="openshift-image-registry/node-ca-pbnwh" Jul 24 14:56:09.239544 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238619 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-kubelet\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.239544 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238633 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-cni-netd\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.239544 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.238698 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96ce8f23-8457-4b01-bde6-4caf871479b2-etc-kubernetes\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.243548 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.243528 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 14:56:09.243602 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.243557 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 14:56:09.243602 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.243571 2531 projected.go:196] Error preparing data for projected volume kube-api-access-z4rz7 for pod openshift-network-diagnostics/network-check-target-mjqx9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 14:56:09.243678 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.243637 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7 podName:ec175d02-f2a5-4492-8133-e642a367da32 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:09.743620116 +0000 UTC m=+1.974096787 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-z4rz7" (UniqueName: "kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7") pod "network-check-target-mjqx9" (UID: "ec175d02-f2a5-4492-8133-e642a367da32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 14:56:09.244429 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.244394 2531 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 24 14:56:09.247521 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.247495 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxcst\" (UniqueName: \"kubernetes.io/projected/91255a26-e33b-408d-81d1-b8ec24d1e7d6-kube-api-access-cxcst\") pod \"node-ca-pbnwh\" (UID: \"91255a26-e33b-408d-81d1-b8ec24d1e7d6\") " pod="openshift-image-registry/node-ca-pbnwh" Jul 24 14:56:09.247600 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.247536 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zqf5\" (UniqueName: \"kubernetes.io/projected/a9494b1d-f2bb-408b-8283-7d622ea5e450-kube-api-access-5zqf5\") pod \"network-metrics-daemon-xgswn\" (UID: \"a9494b1d-f2bb-408b-8283-7d622ea5e450\") " pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:09.247600 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.247546 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzjcr\" (UniqueName: \"kubernetes.io/projected/96ce8f23-8457-4b01-bde6-4caf871479b2-kube-api-access-fzjcr\") pod \"multus-ff2nx\" (UID: \"96ce8f23-8457-4b01-bde6-4caf871479b2\") " pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.247600 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.247554 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prhh6\" (UniqueName: \"kubernetes.io/projected/bdaadab4-2a42-46d7-8ca6-24881a9d54d0-kube-api-access-prhh6\") pod \"node-resolver-hcxz7\" (UID: \"bdaadab4-2a42-46d7-8ca6-24881a9d54d0\") " pod="openshift-dns/node-resolver-hcxz7" Jul 24 14:56:09.339042 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339014 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/447552f9-3427-4e02-9ad1-fa205e53644f-ovnkube-script-lib\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.339221 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339051 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8w4v\" (UniqueName: \"kubernetes.io/projected/447552f9-3427-4e02-9ad1-fa205e53644f-kube-api-access-p8w4v\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.339221 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339074 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02657d23-d776-4da0-939a-e1f8893c5904-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.339221 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339094 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-run-systemd\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.339221 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339139 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-node-log\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.339221 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339162 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/447552f9-3427-4e02-9ad1-fa205e53644f-env-overrides\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.339221 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339163 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-run-systemd\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.339221 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339184 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/447552f9-3427-4e02-9ad1-fa205e53644f-ovn-node-metrics-cert\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.339221 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339212 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02657d23-d776-4da0-939a-e1f8893c5904-os-release\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.339221 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339219 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-node-log\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.339667 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339235 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret\") pod \"global-pull-secret-syncer-6jf8n\" (UID: \"441a2a3d-5df6-4a88-92cc-63d18ba4310f\") " pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:09.339667 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339244 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02657d23-d776-4da0-939a-e1f8893c5904-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.339667 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339257 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6gzs\" (UniqueName: \"kubernetes.io/projected/9c1cadc9-4ba8-451f-b647-22451763a4cc-kube-api-access-b6gzs\") pod \"iptables-alerter-s4h5d\" (UID: \"9c1cadc9-4ba8-451f-b647-22451763a4cc\") " pod="openshift-network-operator/iptables-alerter-s4h5d" Jul 24 14:56:09.339667 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339279 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02657d23-d776-4da0-939a-e1f8893c5904-os-release\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.339667 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339283 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.339667 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339324 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.339667 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.339331 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 14:56:09.339667 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339326 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4c7fe7d-701d-40f5-aafb-7608ab866182-socket-dir\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.339667 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339367 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4c7fe7d-701d-40f5-aafb-7608ab866182-registration-dir\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.339667 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.339402 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret podName:441a2a3d-5df6-4a88-92cc-63d18ba4310f nodeName:}" failed. No retries permitted until 2026-07-24 14:56:09.839382566 +0000 UTC m=+2.069859245 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret") pod "global-pull-secret-syncer-6jf8n" (UID: "441a2a3d-5df6-4a88-92cc-63d18ba4310f") : object "kube-system"/"original-pull-secret" not registered Jul 24 14:56:09.339667 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339423 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c4c7fe7d-701d-40f5-aafb-7608ab866182-socket-dir\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.339667 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339428 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c4c7fe7d-701d-40f5-aafb-7608ab866182-registration-dir\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.339667 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339435 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4c7fe7d-701d-40f5-aafb-7608ab866182-device-dir\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.339667 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339477 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c4c7fe7d-701d-40f5-aafb-7608ab866182-device-dir\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.339667 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339495 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-kubelet\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.339667 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339521 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-cni-netd\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.340477 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339544 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8zzm\" (UniqueName: \"kubernetes.io/projected/c4c7fe7d-701d-40f5-aafb-7608ab866182-kube-api-access-w8zzm\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.340477 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339570 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02657d23-d776-4da0-939a-e1f8893c5904-cnibin\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.340477 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339595 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-var-lib-kubelet\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.340477 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339599 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-kubelet\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.340477 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339619 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-etc-openvswitch\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.340477 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339629 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02657d23-d776-4da0-939a-e1f8893c5904-cnibin\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.340477 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339595 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-cni-netd\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.340477 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339656 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-cni-bin\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.340477 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339636 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/447552f9-3427-4e02-9ad1-fa205e53644f-env-overrides\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.340477 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339681 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/447552f9-3427-4e02-9ad1-fa205e53644f-ovnkube-script-lib\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.340477 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339690 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-var-lib-kubelet\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.340477 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339694 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-etc-openvswitch\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.340477 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339712 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-cni-bin\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.340477 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339713 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-etc-sysctl-d\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.340477 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339745 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-etc-systemd\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.340477 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339773 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/441a2a3d-5df6-4a88-92cc-63d18ba4310f-kubelet-config\") pod \"global-pull-secret-syncer-6jf8n\" (UID: \"441a2a3d-5df6-4a88-92cc-63d18ba4310f\") " pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:09.340477 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339797 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/447552f9-3427-4e02-9ad1-fa205e53644f-ovnkube-config\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.341423 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339805 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-etc-sysctl-d\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.341423 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339821 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-sys\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.341423 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339834 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/441a2a3d-5df6-4a88-92cc-63d18ba4310f-kubelet-config\") pod \"global-pull-secret-syncer-6jf8n\" (UID: \"441a2a3d-5df6-4a88-92cc-63d18ba4310f\") " pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:09.341423 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339807 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-etc-systemd\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.341423 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339850 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-lib-modules\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.341423 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339886 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-sys\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.341423 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339906 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a82a6065-127d-4ced-9487-59b5c2849dbc-agent-certs\") pod \"konnectivity-agent-fcmhs\" (UID: \"a82a6065-127d-4ced-9487-59b5c2849dbc\") " pod="kube-system/konnectivity-agent-fcmhs" Jul 24 14:56:09.341423 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.339942 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-lib-modules\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.341423 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340012 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-systemd-units\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.341423 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340121 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-var-lib-openvswitch\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.341423 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340145 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4c7fe7d-701d-40f5-aafb-7608ab866182-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.341423 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340171 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02657d23-d776-4da0-939a-e1f8893c5904-cni-binary-copy\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.341423 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340195 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-etc-kubernetes\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.341423 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340219 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-var-lib-openvswitch\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.341423 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340231 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c4c7fe7d-701d-40f5-aafb-7608ab866182-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.341423 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340244 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/447552f9-3427-4e02-9ad1-fa205e53644f-ovnkube-config\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.341423 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340268 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02657d23-d776-4da0-939a-e1f8893c5904-system-cni-dir\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.342322 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340046 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-systemd-units\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.342322 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340232 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02657d23-d776-4da0-939a-e1f8893c5904-system-cni-dir\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.342322 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340278 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-etc-kubernetes\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.342322 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340306 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-run\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.342322 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340331 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fde578a4-ae0f-425f-991c-30917584906f-etc-tuned\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.342322 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340355 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/441a2a3d-5df6-4a88-92cc-63d18ba4310f-dbus\") pod \"global-pull-secret-syncer-6jf8n\" (UID: \"441a2a3d-5df6-4a88-92cc-63d18ba4310f\") " pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:09.342322 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340374 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-run\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.342322 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340380 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-run-netns\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.342322 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340434 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-run-ovn\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.342322 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340463 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-run-openvswitch\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.342322 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340485 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/441a2a3d-5df6-4a88-92cc-63d18ba4310f-dbus\") pod \"global-pull-secret-syncer-6jf8n\" (UID: \"441a2a3d-5df6-4a88-92cc-63d18ba4310f\") " pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:09.342322 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340490 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4c7fe7d-701d-40f5-aafb-7608ab866182-etc-selinux\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.342322 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340526 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a82a6065-127d-4ced-9487-59b5c2849dbc-konnectivity-ca\") pod \"konnectivity-agent-fcmhs\" (UID: \"a82a6065-127d-4ced-9487-59b5c2849dbc\") " pod="kube-system/konnectivity-agent-fcmhs" Jul 24 14:56:09.342322 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340556 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fde578a4-ae0f-425f-991c-30917584906f-tmp\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.342322 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340571 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c4c7fe7d-701d-40f5-aafb-7608ab866182-etc-selinux\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.342322 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340583 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/02657d23-d776-4da0-939a-e1f8893c5904-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.342322 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340612 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrr9b\" (UniqueName: \"kubernetes.io/projected/02657d23-d776-4da0-939a-e1f8893c5904-kube-api-access-qrr9b\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.342946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340617 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-run-ovn\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.342946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340635 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-host\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.342946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340654 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-run-openvswitch\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.342946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340715 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-run-ovn-kubernetes\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.342946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340661 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-run-ovn-kubernetes\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.342946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340757 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4c7fe7d-701d-40f5-aafb-7608ab866182-sys-fs\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.342946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340784 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9c1cadc9-4ba8-451f-b647-22451763a4cc-iptables-alerter-script\") pod \"iptables-alerter-s4h5d\" (UID: \"9c1cadc9-4ba8-451f-b647-22451763a4cc\") " pod="openshift-network-operator/iptables-alerter-s4h5d" Jul 24 14:56:09.342946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340799 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02657d23-d776-4da0-939a-e1f8893c5904-cni-binary-copy\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.342946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340843 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c4c7fe7d-701d-40f5-aafb-7608ab866182-sys-fs\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.342946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340887 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-run-netns\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.342946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340901 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-slash\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.342946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340932 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-log-socket\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.342946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340961 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/02657d23-d776-4da0-939a-e1f8893c5904-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.342946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.340988 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-etc-modprobe-d\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.342946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.341012 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-etc-sysconfig\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.342946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.341044 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m72hb\" (UniqueName: \"kubernetes.io/projected/fde578a4-ae0f-425f-991c-30917584906f-kube-api-access-m72hb\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.342946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.341068 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c1cadc9-4ba8-451f-b647-22451763a4cc-host-slash\") pod \"iptables-alerter-s4h5d\" (UID: \"9c1cadc9-4ba8-451f-b647-22451763a4cc\") " pod="openshift-network-operator/iptables-alerter-s4h5d" Jul 24 14:56:09.343526 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.341118 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-etc-sysctl-conf\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.343526 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.341219 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/02657d23-d776-4da0-939a-e1f8893c5904-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.343526 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.341244 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-etc-sysctl-conf\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.343526 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.341321 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-host\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.343526 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.341337 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-etc-modprobe-d\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.343526 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.341352 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9c1cadc9-4ba8-451f-b647-22451763a4cc-iptables-alerter-script\") pod \"iptables-alerter-s4h5d\" (UID: \"9c1cadc9-4ba8-451f-b647-22451763a4cc\") " pod="openshift-network-operator/iptables-alerter-s4h5d" Jul 24 14:56:09.343526 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.341413 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c1cadc9-4ba8-451f-b647-22451763a4cc-host-slash\") pod \"iptables-alerter-s4h5d\" (UID: \"9c1cadc9-4ba8-451f-b647-22451763a4cc\") " pod="openshift-network-operator/iptables-alerter-s4h5d" Jul 24 14:56:09.343526 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.341457 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-host-slash\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.343526 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.341496 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/447552f9-3427-4e02-9ad1-fa205e53644f-log-socket\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.343526 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.341552 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fde578a4-ae0f-425f-991c-30917584906f-etc-sysconfig\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.343526 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.341711 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/02657d23-d776-4da0-939a-e1f8893c5904-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.343526 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.341835 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a82a6065-127d-4ced-9487-59b5c2849dbc-konnectivity-ca\") pod \"konnectivity-agent-fcmhs\" (UID: \"a82a6065-127d-4ced-9487-59b5c2849dbc\") " pod="kube-system/konnectivity-agent-fcmhs" Jul 24 14:56:09.343526 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.341953 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/447552f9-3427-4e02-9ad1-fa205e53644f-ovn-node-metrics-cert\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.343526 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.342621 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a82a6065-127d-4ced-9487-59b5c2849dbc-agent-certs\") pod \"konnectivity-agent-fcmhs\" (UID: \"a82a6065-127d-4ced-9487-59b5c2849dbc\") " pod="kube-system/konnectivity-agent-fcmhs" Jul 24 14:56:09.343526 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.343381 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fde578a4-ae0f-425f-991c-30917584906f-tmp\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.344081 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.343544 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fde578a4-ae0f-425f-991c-30917584906f-etc-tuned\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.352416 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.352396 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrr9b\" (UniqueName: \"kubernetes.io/projected/02657d23-d776-4da0-939a-e1f8893c5904-kube-api-access-qrr9b\") pod \"multus-additional-cni-plugins-vgqjg\" (UID: \"02657d23-d776-4da0-939a-e1f8893c5904\") " pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.352655 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.352641 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8w4v\" (UniqueName: \"kubernetes.io/projected/447552f9-3427-4e02-9ad1-fa205e53644f-kube-api-access-p8w4v\") pod \"ovnkube-node-f74fd\" (UID: \"447552f9-3427-4e02-9ad1-fa205e53644f\") " pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.352905 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.352885 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8zzm\" (UniqueName: \"kubernetes.io/projected/c4c7fe7d-701d-40f5-aafb-7608ab866182-kube-api-access-w8zzm\") pod \"aws-ebs-csi-driver-node-7bsxq\" (UID: \"c4c7fe7d-701d-40f5-aafb-7608ab866182\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.352963 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.352948 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6gzs\" (UniqueName: \"kubernetes.io/projected/9c1cadc9-4ba8-451f-b647-22451763a4cc-kube-api-access-b6gzs\") pod \"iptables-alerter-s4h5d\" (UID: \"9c1cadc9-4ba8-451f-b647-22451763a4cc\") " pod="openshift-network-operator/iptables-alerter-s4h5d" Jul 24 14:56:09.353050 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.353037 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m72hb\" (UniqueName: \"kubernetes.io/projected/fde578a4-ae0f-425f-991c-30917584906f-kube-api-access-m72hb\") pod \"tuned-62nf6\" (UID: \"fde578a4-ae0f-425f-991c-30917584906f\") " pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.440748 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:09.440710 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd99284ebfc3675b576b1ca8cc30be861.slice/crio-a0dea943c7a3dc09a66366eceb6d6e148e38c03bece3552308392c214ded50d0 WatchSource:0}: Error finding container a0dea943c7a3dc09a66366eceb6d6e148e38c03bece3552308392c214ded50d0: Status 404 returned error can't find the container with id a0dea943c7a3dc09a66366eceb6d6e148e38c03bece3552308392c214ded50d0 Jul 24 14:56:09.444196 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.444180 2531 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 14:56:09.451828 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.451809 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pbnwh" Jul 24 14:56:09.458188 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.458171 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fcmhs" Jul 24 14:56:09.459430 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:09.459411 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91255a26_e33b_408d_81d1_b8ec24d1e7d6.slice/crio-0276b194eed154c506216f5e4bee562f1ba61f157fb47e3cb6ef9fddee5949c8 WatchSource:0}: Error finding container 0276b194eed154c506216f5e4bee562f1ba61f157fb47e3cb6ef9fddee5949c8: Status 404 returned error can't find the container with id 0276b194eed154c506216f5e4bee562f1ba61f157fb47e3cb6ef9fddee5949c8 Jul 24 14:56:09.465298 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:09.465279 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda82a6065_127d_4ced_9487_59b5c2849dbc.slice/crio-20fba469c5191ae8b3580279e8110bce5a02788cad74284bfa4e71558b223500 WatchSource:0}: Error finding container 20fba469c5191ae8b3580279e8110bce5a02788cad74284bfa4e71558b223500: Status 404 returned error can't find the container with id 20fba469c5191ae8b3580279e8110bce5a02788cad74284bfa4e71558b223500 Jul 24 14:56:09.475078 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.475061 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hcxz7" Jul 24 14:56:09.482786 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:09.482770 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdaadab4_2a42_46d7_8ca6_24881a9d54d0.slice/crio-1ddcc00152d23a60f3e00b82c4e2c4465a6eb86c9cadb1918b1ad4dc73b720b5 WatchSource:0}: Error finding container 1ddcc00152d23a60f3e00b82c4e2c4465a6eb86c9cadb1918b1ad4dc73b720b5: Status 404 returned error can't find the container with id 1ddcc00152d23a60f3e00b82c4e2c4465a6eb86c9cadb1918b1ad4dc73b720b5 Jul 24 14:56:09.484609 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.484591 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ff2nx" Jul 24 14:56:09.490968 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:09.490942 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96ce8f23_8457_4b01_bde6_4caf871479b2.slice/crio-228a7b1d5ac99a5453fe52b907153a3b066bdbe759ef8a9ccd4c326948e09ac0 WatchSource:0}: Error finding container 228a7b1d5ac99a5453fe52b907153a3b066bdbe759ef8a9ccd4c326948e09ac0: Status 404 returned error can't find the container with id 228a7b1d5ac99a5453fe52b907153a3b066bdbe759ef8a9ccd4c326948e09ac0 Jul 24 14:56:09.497119 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.497101 2531 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 14:56:09.503821 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.503805 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s4h5d" Jul 24 14:56:09.510645 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:09.510624 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c1cadc9_4ba8_451f_b647_22451763a4cc.slice/crio-9e10a2d4416ff02de698bbf42dcd40cbb7d080cb6589b4abdb6aef156fc9636c WatchSource:0}: Error finding container 9e10a2d4416ff02de698bbf42dcd40cbb7d080cb6589b4abdb6aef156fc9636c: Status 404 returned error can't find the container with id 9e10a2d4416ff02de698bbf42dcd40cbb7d080cb6589b4abdb6aef156fc9636c Jul 24 14:56:09.516027 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.516010 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-62nf6" Jul 24 14:56:09.519316 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.519302 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:09.520901 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.520859 2531 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 14:56:09.522444 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:09.522426 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfde578a4_ae0f_425f_991c_30917584906f.slice/crio-3820a1e2b97b989aa74eac1923d114005087802d4d227a6b4f110302306572f1 WatchSource:0}: Error finding container 3820a1e2b97b989aa74eac1923d114005087802d4d227a6b4f110302306572f1: Status 404 returned error can't find the container with id 3820a1e2b97b989aa74eac1923d114005087802d4d227a6b4f110302306572f1 Jul 24 14:56:09.526557 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:09.526537 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod447552f9_3427_4e02_9ad1_fa205e53644f.slice/crio-efae4f95d5b4a72be782cc98d326089229d4e6c4b7efde39c94bd3c736f5f48b WatchSource:0}: Error finding container efae4f95d5b4a72be782cc98d326089229d4e6c4b7efde39c94bd3c736f5f48b: Status 404 returned error can't find the container with id efae4f95d5b4a72be782cc98d326089229d4e6c4b7efde39c94bd3c736f5f48b Jul 24 14:56:09.545030 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.545012 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" Jul 24 14:56:09.555411 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.555386 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vgqjg" Jul 24 14:56:09.558302 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:09.558285 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4c7fe7d_701d_40f5_aafb_7608ab866182.slice/crio-7b16348636397fe5ee2efb3ed73c82edb469f7f5f95573af2d6b485874262ff4 WatchSource:0}: Error finding container 7b16348636397fe5ee2efb3ed73c82edb469f7f5f95573af2d6b485874262ff4: Status 404 returned error can't find the container with id 7b16348636397fe5ee2efb3ed73c82edb469f7f5f95573af2d6b485874262ff4 Jul 24 14:56:09.558855 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:09.558815 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd97724e3608d55c7f74ae763e45fb90.slice/crio-d67690634296706e3bdd4262f8810f10c40aaa08def66f547b94967205690772 WatchSource:0}: Error finding container d67690634296706e3bdd4262f8810f10c40aaa08def66f547b94967205690772: Status 404 returned error can't find the container with id d67690634296706e3bdd4262f8810f10c40aaa08def66f547b94967205690772 Jul 24 14:56:09.563611 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:09.563591 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02657d23_d776_4da0_939a_e1f8893c5904.slice/crio-45954a3d1dab65cb49e6b786cbb803f370dc82c4d5e3ade0e59779c0bd9b1be9 WatchSource:0}: Error finding container 45954a3d1dab65cb49e6b786cbb803f370dc82c4d5e3ade0e59779c0bd9b1be9: Status 404 returned error can't find the container with id 45954a3d1dab65cb49e6b786cbb803f370dc82c4d5e3ade0e59779c0bd9b1be9 Jul 24 14:56:09.565132 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.565093 2531 kuberuntime_manager.go:1459] "Unhandled Error" err="init container egress-router-binary-copy start failed in pod multus-additional-cni-plugins-vgqjg_openshift-multus(02657d23-d776-4da0-939a-e1f8893c5904): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 24 14:56:09.565214 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.565137 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"egress-router-binary-copy\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-multus/multus-additional-cni-plugins-vgqjg" podUID="02657d23-d776-4da0-939a-e1f8893c5904" Jul 24 14:56:09.747330 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.745433 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs\") pod \"network-metrics-daemon-xgswn\" (UID: \"a9494b1d-f2bb-408b-8283-7d622ea5e450\") " pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:09.747330 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.745557 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4rz7\" (UniqueName: \"kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7\") pod \"network-check-target-mjqx9\" (UID: \"ec175d02-f2a5-4492-8133-e642a367da32\") " pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:09.747330 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.745703 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 14:56:09.747330 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.745723 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 14:56:09.747330 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.745744 2531 projected.go:196] Error preparing data for projected volume kube-api-access-z4rz7 for pod openshift-network-diagnostics/network-check-target-mjqx9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 14:56:09.747330 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.745804 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7 podName:ec175d02-f2a5-4492-8133-e642a367da32 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:10.745780703 +0000 UTC m=+2.976257382 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-z4rz7" (UniqueName: "kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7") pod "network-check-target-mjqx9" (UID: "ec175d02-f2a5-4492-8133-e642a367da32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 14:56:09.747330 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.746284 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 14:56:09.747330 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.746342 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs podName:a9494b1d-f2bb-408b-8283-7d622ea5e450 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:10.746325349 +0000 UTC m=+2.976802017 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs") pod "network-metrics-daemon-xgswn" (UID: "a9494b1d-f2bb-408b-8283-7d622ea5e450") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 14:56:09.846812 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:09.846785 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret\") pod \"global-pull-secret-syncer-6jf8n\" (UID: \"441a2a3d-5df6-4a88-92cc-63d18ba4310f\") " pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:09.846932 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.846900 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 14:56:09.846982 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:09.846946 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret podName:441a2a3d-5df6-4a88-92cc-63d18ba4310f nodeName:}" failed. No retries permitted until 2026-07-24 14:56:10.846934479 +0000 UTC m=+3.077411144 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret") pod "global-pull-secret-syncer-6jf8n" (UID: "441a2a3d-5df6-4a88-92cc-63d18ba4310f") : object "kube-system"/"original-pull-secret" not registered Jul 24 14:56:10.123264 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:10.123185 2531 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 14:56:10.176840 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:10.176770 2531 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 14:51:10 +0000 UTC" deadline="2028-02-14 23:42:42.098588337 +0000 UTC" Jul 24 14:56:10.176840 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:10.176837 2531 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13688h46m31.921756099s" Jul 24 14:56:10.273684 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:10.273652 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:10.273875 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:10.273795 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xgswn" podUID="a9494b1d-f2bb-408b-8283-7d622ea5e450" Jul 24 14:56:10.274259 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:10.274238 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:10.274362 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:10.274341 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mjqx9" podUID="ec175d02-f2a5-4492-8133-e642a367da32" Jul 24 14:56:10.286434 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:10.286378 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-62nf6" event={"ID":"fde578a4-ae0f-425f-991c-30917584906f","Type":"ContainerStarted","Data":"3820a1e2b97b989aa74eac1923d114005087802d4d227a6b4f110302306572f1"} Jul 24 14:56:10.298059 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:10.297846 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ff2nx" event={"ID":"96ce8f23-8457-4b01-bde6-4caf871479b2","Type":"ContainerStarted","Data":"228a7b1d5ac99a5453fe52b907153a3b066bdbe759ef8a9ccd4c326948e09ac0"} Jul 24 14:56:10.305100 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:10.305052 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vgqjg" event={"ID":"02657d23-d776-4da0-939a-e1f8893c5904","Type":"ContainerStarted","Data":"45954a3d1dab65cb49e6b786cbb803f370dc82c4d5e3ade0e59779c0bd9b1be9"} Jul 24 14:56:10.309322 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:10.309278 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal" event={"ID":"fd97724e3608d55c7f74ae763e45fb90","Type":"ContainerStarted","Data":"d67690634296706e3bdd4262f8810f10c40aaa08def66f547b94967205690772"} Jul 24 14:56:10.311309 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:10.311275 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"egress-router-binary-copy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:53008d8e30cdba8a421c865d7fd13497538e84d6843c1692df37914257440e91\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-multus/multus-additional-cni-plugins-vgqjg" podUID="02657d23-d776-4da0-939a-e1f8893c5904" Jul 24 14:56:10.317645 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:10.317615 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" event={"ID":"c4c7fe7d-701d-40f5-aafb-7608ab866182","Type":"ContainerStarted","Data":"7b16348636397fe5ee2efb3ed73c82edb469f7f5f95573af2d6b485874262ff4"} Jul 24 14:56:10.330433 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:10.330403 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s4h5d" event={"ID":"9c1cadc9-4ba8-451f-b647-22451763a4cc","Type":"ContainerStarted","Data":"9e10a2d4416ff02de698bbf42dcd40cbb7d080cb6589b4abdb6aef156fc9636c"} Jul 24 14:56:10.338382 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:10.338360 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hcxz7" event={"ID":"bdaadab4-2a42-46d7-8ca6-24881a9d54d0","Type":"ContainerStarted","Data":"1ddcc00152d23a60f3e00b82c4e2c4465a6eb86c9cadb1918b1ad4dc73b720b5"} Jul 24 14:56:10.343330 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:10.343303 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fcmhs" event={"ID":"a82a6065-127d-4ced-9487-59b5c2849dbc","Type":"ContainerStarted","Data":"20fba469c5191ae8b3580279e8110bce5a02788cad74284bfa4e71558b223500"} Jul 24 14:56:10.347516 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:10.347493 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pbnwh" event={"ID":"91255a26-e33b-408d-81d1-b8ec24d1e7d6","Type":"ContainerStarted","Data":"0276b194eed154c506216f5e4bee562f1ba61f157fb47e3cb6ef9fddee5949c8"} Jul 24 14:56:10.352969 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:10.352934 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" event={"ID":"d99284ebfc3675b576b1ca8cc30be861","Type":"ContainerStarted","Data":"a0dea943c7a3dc09a66366eceb6d6e148e38c03bece3552308392c214ded50d0"} Jul 24 14:56:10.360629 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:10.360605 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" event={"ID":"447552f9-3427-4e02-9ad1-fa205e53644f","Type":"ContainerStarted","Data":"efae4f95d5b4a72be782cc98d326089229d4e6c4b7efde39c94bd3c736f5f48b"} Jul 24 14:56:10.754852 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:10.754811 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4rz7\" (UniqueName: \"kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7\") pod \"network-check-target-mjqx9\" (UID: \"ec175d02-f2a5-4492-8133-e642a367da32\") " pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:10.755071 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:10.754879 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs\") pod \"network-metrics-daemon-xgswn\" (UID: \"a9494b1d-f2bb-408b-8283-7d622ea5e450\") " pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:10.755071 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:10.755032 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 14:56:10.755192 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:10.755104 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs podName:a9494b1d-f2bb-408b-8283-7d622ea5e450 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:12.755084313 +0000 UTC m=+4.985560988 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs") pod "network-metrics-daemon-xgswn" (UID: "a9494b1d-f2bb-408b-8283-7d622ea5e450") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 14:56:10.755583 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:10.755554 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 14:56:10.755665 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:10.755585 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 14:56:10.755665 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:10.755598 2531 projected.go:196] Error preparing data for projected volume kube-api-access-z4rz7 for pod openshift-network-diagnostics/network-check-target-mjqx9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 14:56:10.755665 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:10.755646 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7 podName:ec175d02-f2a5-4492-8133-e642a367da32 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:12.755629439 +0000 UTC m=+4.986106107 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-z4rz7" (UniqueName: "kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7") pod "network-check-target-mjqx9" (UID: "ec175d02-f2a5-4492-8133-e642a367da32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 14:56:10.855489 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:10.855447 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret\") pod \"global-pull-secret-syncer-6jf8n\" (UID: \"441a2a3d-5df6-4a88-92cc-63d18ba4310f\") " pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:10.855685 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:10.855625 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 14:56:10.855757 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:10.855691 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret podName:441a2a3d-5df6-4a88-92cc-63d18ba4310f nodeName:}" failed. No retries permitted until 2026-07-24 14:56:12.855670454 +0000 UTC m=+5.086147120 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret") pod "global-pull-secret-syncer-6jf8n" (UID: "441a2a3d-5df6-4a88-92cc-63d18ba4310f") : object "kube-system"/"original-pull-secret" not registered Jul 24 14:56:11.179855 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:11.177689 2531 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 14:51:10 +0000 UTC" deadline="2028-04-22 12:14:06.666550704 +0000 UTC" Jul 24 14:56:11.179855 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:11.177738 2531 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15309h17m55.48881874s" Jul 24 14:56:11.270783 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:11.270747 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:11.271003 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:11.270882 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jf8n" podUID="441a2a3d-5df6-4a88-92cc-63d18ba4310f" Jul 24 14:56:11.800077 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:11.800025 2531 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 14:56:12.277009 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:12.273216 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:12.277009 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:12.273360 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xgswn" podUID="a9494b1d-f2bb-408b-8283-7d622ea5e450" Jul 24 14:56:12.277009 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:12.273742 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:12.277009 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:12.273848 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mjqx9" podUID="ec175d02-f2a5-4492-8133-e642a367da32" Jul 24 14:56:12.773723 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:12.773631 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4rz7\" (UniqueName: \"kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7\") pod \"network-check-target-mjqx9\" (UID: \"ec175d02-f2a5-4492-8133-e642a367da32\") " pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:12.773723 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:12.773703 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs\") pod \"network-metrics-daemon-xgswn\" (UID: \"a9494b1d-f2bb-408b-8283-7d622ea5e450\") " pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:12.773997 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:12.773864 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 14:56:12.774068 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:12.774047 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 14:56:12.774122 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:12.774069 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 14:56:12.774122 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:12.774083 2531 projected.go:196] Error preparing data for projected volume kube-api-access-z4rz7 for pod openshift-network-diagnostics/network-check-target-mjqx9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 14:56:12.774221 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:12.774141 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7 podName:ec175d02-f2a5-4492-8133-e642a367da32 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:16.774118962 +0000 UTC m=+9.004595650 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-z4rz7" (UniqueName: "kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7") pod "network-check-target-mjqx9" (UID: "ec175d02-f2a5-4492-8133-e642a367da32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 14:56:12.774716 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:12.774661 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs podName:a9494b1d-f2bb-408b-8283-7d622ea5e450 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:16.774627452 +0000 UTC m=+9.005104130 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs") pod "network-metrics-daemon-xgswn" (UID: "a9494b1d-f2bb-408b-8283-7d622ea5e450") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 14:56:12.874328 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:12.874282 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret\") pod \"global-pull-secret-syncer-6jf8n\" (UID: \"441a2a3d-5df6-4a88-92cc-63d18ba4310f\") " pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:12.874499 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:12.874470 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 14:56:12.874561 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:12.874531 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret podName:441a2a3d-5df6-4a88-92cc-63d18ba4310f nodeName:}" failed. No retries permitted until 2026-07-24 14:56:16.874513527 +0000 UTC m=+9.104990193 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret") pod "global-pull-secret-syncer-6jf8n" (UID: "441a2a3d-5df6-4a88-92cc-63d18ba4310f") : object "kube-system"/"original-pull-secret" not registered Jul 24 14:56:13.271818 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:13.271568 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:13.271818 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:13.271711 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jf8n" podUID="441a2a3d-5df6-4a88-92cc-63d18ba4310f" Jul 24 14:56:14.271090 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:14.271050 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:14.271603 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:14.271209 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xgswn" podUID="a9494b1d-f2bb-408b-8283-7d622ea5e450" Jul 24 14:56:14.274007 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:14.273845 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:14.274291 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:14.274239 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mjqx9" podUID="ec175d02-f2a5-4492-8133-e642a367da32" Jul 24 14:56:15.270615 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:15.270580 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:15.270821 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:15.270705 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jf8n" podUID="441a2a3d-5df6-4a88-92cc-63d18ba4310f" Jul 24 14:56:16.270985 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:16.270945 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:16.271463 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:16.271099 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xgswn" podUID="a9494b1d-f2bb-408b-8283-7d622ea5e450" Jul 24 14:56:16.271463 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:16.271177 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:16.271463 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:16.271323 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mjqx9" podUID="ec175d02-f2a5-4492-8133-e642a367da32" Jul 24 14:56:16.808466 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:16.808417 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs\") pod \"network-metrics-daemon-xgswn\" (UID: \"a9494b1d-f2bb-408b-8283-7d622ea5e450\") " pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:16.808672 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:16.808551 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4rz7\" (UniqueName: \"kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7\") pod \"network-check-target-mjqx9\" (UID: \"ec175d02-f2a5-4492-8133-e642a367da32\") " pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:16.808742 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:16.808724 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 14:56:16.808961 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:16.808746 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 14:56:16.809053 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:16.809016 2531 projected.go:196] Error preparing data for projected volume kube-api-access-z4rz7 for pod openshift-network-diagnostics/network-check-target-mjqx9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 14:56:16.809143 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:16.809106 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7 podName:ec175d02-f2a5-4492-8133-e642a367da32 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:24.80907085 +0000 UTC m=+17.039547592 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-z4rz7" (UniqueName: "kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7") pod "network-check-target-mjqx9" (UID: "ec175d02-f2a5-4492-8133-e642a367da32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 14:56:16.821797 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:16.821768 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 14:56:16.821965 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:16.821870 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs podName:a9494b1d-f2bb-408b-8283-7d622ea5e450 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:24.821848339 +0000 UTC m=+17.052325019 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs") pod "network-metrics-daemon-xgswn" (UID: "a9494b1d-f2bb-408b-8283-7d622ea5e450") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 14:56:16.910208 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:16.909623 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret\") pod \"global-pull-secret-syncer-6jf8n\" (UID: \"441a2a3d-5df6-4a88-92cc-63d18ba4310f\") " pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:16.910208 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:16.909820 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 14:56:16.910208 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:16.909878 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret podName:441a2a3d-5df6-4a88-92cc-63d18ba4310f nodeName:}" failed. No retries permitted until 2026-07-24 14:56:24.909863677 +0000 UTC m=+17.140340341 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret") pod "global-pull-secret-syncer-6jf8n" (UID: "441a2a3d-5df6-4a88-92cc-63d18ba4310f") : object "kube-system"/"original-pull-secret" not registered Jul 24 14:56:17.272063 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:17.271495 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:17.272063 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:17.271642 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jf8n" podUID="441a2a3d-5df6-4a88-92cc-63d18ba4310f" Jul 24 14:56:18.272346 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:18.272313 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:18.272689 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:18.272420 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mjqx9" podUID="ec175d02-f2a5-4492-8133-e642a367da32" Jul 24 14:56:18.273120 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:18.272924 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:18.273120 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:18.273065 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xgswn" podUID="a9494b1d-f2bb-408b-8283-7d622ea5e450" Jul 24 14:56:19.271242 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:19.271191 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:19.271449 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:19.271349 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jf8n" podUID="441a2a3d-5df6-4a88-92cc-63d18ba4310f" Jul 24 14:56:20.271071 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:20.271028 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:20.271467 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:20.271097 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:20.271467 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:20.271224 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mjqx9" podUID="ec175d02-f2a5-4492-8133-e642a367da32" Jul 24 14:56:20.271467 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:20.271349 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xgswn" podUID="a9494b1d-f2bb-408b-8283-7d622ea5e450" Jul 24 14:56:21.271257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:21.271185 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:21.271678 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:21.271306 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jf8n" podUID="441a2a3d-5df6-4a88-92cc-63d18ba4310f" Jul 24 14:56:22.270891 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:22.270850 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:22.271093 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:22.270957 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:22.271093 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:22.271020 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xgswn" podUID="a9494b1d-f2bb-408b-8283-7d622ea5e450" Jul 24 14:56:22.271179 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:22.271157 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mjqx9" podUID="ec175d02-f2a5-4492-8133-e642a367da32" Jul 24 14:56:23.270784 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:23.270752 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:23.271243 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:23.270919 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jf8n" podUID="441a2a3d-5df6-4a88-92cc-63d18ba4310f" Jul 24 14:56:24.271089 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:24.271055 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:24.271514 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:24.271222 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:24.271514 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:24.271227 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xgswn" podUID="a9494b1d-f2bb-408b-8283-7d622ea5e450" Jul 24 14:56:24.271514 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:24.271333 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mjqx9" podUID="ec175d02-f2a5-4492-8133-e642a367da32" Jul 24 14:56:24.877115 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:24.877082 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4rz7\" (UniqueName: \"kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7\") pod \"network-check-target-mjqx9\" (UID: \"ec175d02-f2a5-4492-8133-e642a367da32\") " pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:24.877297 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:24.877135 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs\") pod \"network-metrics-daemon-xgswn\" (UID: \"a9494b1d-f2bb-408b-8283-7d622ea5e450\") " pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:24.877297 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:24.877233 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 14:56:24.877297 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:24.877262 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 14:56:24.877297 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:24.877289 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 14:56:24.877493 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:24.877303 2531 projected.go:196] Error preparing data for projected volume kube-api-access-z4rz7 for pod openshift-network-diagnostics/network-check-target-mjqx9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 14:56:24.877493 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:24.877290 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs podName:a9494b1d-f2bb-408b-8283-7d622ea5e450 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:40.877271069 +0000 UTC m=+33.107747735 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs") pod "network-metrics-daemon-xgswn" (UID: "a9494b1d-f2bb-408b-8283-7d622ea5e450") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 14:56:24.877493 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:24.877355 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7 podName:ec175d02-f2a5-4492-8133-e642a367da32 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:40.877338939 +0000 UTC m=+33.107815601 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-z4rz7" (UniqueName: "kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7") pod "network-check-target-mjqx9" (UID: "ec175d02-f2a5-4492-8133-e642a367da32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 14:56:24.977778 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:24.977690 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret\") pod \"global-pull-secret-syncer-6jf8n\" (UID: \"441a2a3d-5df6-4a88-92cc-63d18ba4310f\") " pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:24.977963 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:24.977815 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 14:56:24.977963 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:24.977892 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret podName:441a2a3d-5df6-4a88-92cc-63d18ba4310f nodeName:}" failed. No retries permitted until 2026-07-24 14:56:40.977859058 +0000 UTC m=+33.208335723 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret") pod "global-pull-secret-syncer-6jf8n" (UID: "441a2a3d-5df6-4a88-92cc-63d18ba4310f") : object "kube-system"/"original-pull-secret" not registered Jul 24 14:56:25.271382 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:25.271312 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:25.271767 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:25.271444 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jf8n" podUID="441a2a3d-5df6-4a88-92cc-63d18ba4310f" Jul 24 14:56:26.272521 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:26.271829 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:26.272521 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:26.271983 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xgswn" podUID="a9494b1d-f2bb-408b-8283-7d622ea5e450" Jul 24 14:56:26.272521 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:26.272389 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:26.272521 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:26.272485 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mjqx9" podUID="ec175d02-f2a5-4492-8133-e642a367da32" Jul 24 14:56:27.270881 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:27.270832 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:27.271059 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:27.270979 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jf8n" podUID="441a2a3d-5df6-4a88-92cc-63d18ba4310f" Jul 24 14:56:28.272136 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:28.272104 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:28.272587 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:28.272234 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xgswn" podUID="a9494b1d-f2bb-408b-8283-7d622ea5e450" Jul 24 14:56:28.272587 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:28.272305 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:28.272587 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:28.272417 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mjqx9" podUID="ec175d02-f2a5-4492-8133-e642a367da32" Jul 24 14:56:29.270760 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:29.270721 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:29.270979 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:29.270851 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jf8n" podUID="441a2a3d-5df6-4a88-92cc-63d18ba4310f" Jul 24 14:56:30.271650 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:30.271618 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:30.272177 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:30.271666 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:30.272177 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:30.271751 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xgswn" podUID="a9494b1d-f2bb-408b-8283-7d622ea5e450" Jul 24 14:56:30.272177 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:30.271881 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mjqx9" podUID="ec175d02-f2a5-4492-8133-e642a367da32" Jul 24 14:56:31.271283 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.271105 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:31.271432 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:31.271369 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jf8n" podUID="441a2a3d-5df6-4a88-92cc-63d18ba4310f" Jul 24 14:56:31.417833 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.417759 2531 generic.go:356] "Generic (PLEG): container finished" podID="02657d23-d776-4da0-939a-e1f8893c5904" containerID="b1dd43b9f6447257b3a9e9f1f3f6da7dcbb1eeffb66c12bde2fe4ad50b34483e" exitCode=0 Jul 24 14:56:31.418679 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.417833 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vgqjg" event={"ID":"02657d23-d776-4da0-939a-e1f8893c5904","Type":"ContainerDied","Data":"b1dd43b9f6447257b3a9e9f1f3f6da7dcbb1eeffb66c12bde2fe4ad50b34483e"} Jul 24 14:56:31.419457 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.419312 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal" event={"ID":"fd97724e3608d55c7f74ae763e45fb90","Type":"ContainerStarted","Data":"5d0a556ac5016dad4c2e8f0e9c65ac3c840328b108f98e6ec5873582b20b816b"} Jul 24 14:56:31.421468 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.420983 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" event={"ID":"c4c7fe7d-701d-40f5-aafb-7608ab866182","Type":"ContainerStarted","Data":"a9b54b4a3708bc93d582da44b8896ca23a0d1d41c2e280d5b7389c8bd11bbeeb"} Jul 24 14:56:31.424030 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.424010 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hcxz7" event={"ID":"bdaadab4-2a42-46d7-8ca6-24881a9d54d0","Type":"ContainerStarted","Data":"8d1cd5f5d950fafb2ace31419572fb0ee8416c47bc4d6f5f18837a415c48fe8f"} Jul 24 14:56:31.425310 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.425289 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fcmhs" event={"ID":"a82a6065-127d-4ced-9487-59b5c2849dbc","Type":"ContainerStarted","Data":"1995ca30599b882426e80335656360f556e45dd324fd5692cf203601e4077978"} Jul 24 14:56:31.426620 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.426601 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pbnwh" event={"ID":"91255a26-e33b-408d-81d1-b8ec24d1e7d6","Type":"ContainerStarted","Data":"f18c9b6135bf3036ea3b9eaeb5e62dd2639b5ed3bff67fcacbc1939ae7866d7d"} Jul 24 14:56:31.427959 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.427940 2531 generic.go:356] "Generic (PLEG): container finished" podID="d99284ebfc3675b576b1ca8cc30be861" containerID="727fe40374551261a38888e71fe4f9d8ea2b169ab40f6f523fca45b44ccd55d1" exitCode=0 Jul 24 14:56:31.428058 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.427976 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" event={"ID":"d99284ebfc3675b576b1ca8cc30be861","Type":"ContainerDied","Data":"727fe40374551261a38888e71fe4f9d8ea2b169ab40f6f523fca45b44ccd55d1"} Jul 24 14:56:31.431596 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.431576 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" event={"ID":"447552f9-3427-4e02-9ad1-fa205e53644f","Type":"ContainerStarted","Data":"222db629037d0788abc82aceeb8e48527f7adbd8125db9c9880767d4b43a3330"} Jul 24 14:56:31.431700 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.431630 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" event={"ID":"447552f9-3427-4e02-9ad1-fa205e53644f","Type":"ContainerStarted","Data":"ac971bd0e9159d1ce2d55362a3b93e6bd2bdce3c9ce51add1340f8bb02a043b5"} Jul 24 14:56:31.431700 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.431674 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" event={"ID":"447552f9-3427-4e02-9ad1-fa205e53644f","Type":"ContainerStarted","Data":"6dabb1a9e39926e30e51c3b64ed0b965ab8db390362aa1e830eeffbe8b15f688"} Jul 24 14:56:31.431782 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.431741 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" event={"ID":"447552f9-3427-4e02-9ad1-fa205e53644f","Type":"ContainerStarted","Data":"efa3124dd197de10973492c409c889eac9f5d2ce218bd5489a5016c0be8cbceb"} Jul 24 14:56:31.431834 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.431784 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" event={"ID":"447552f9-3427-4e02-9ad1-fa205e53644f","Type":"ContainerStarted","Data":"cf153f6c733e1e34cebfa43380fe110043eb72b7af97854477094eef2b6dfeeb"} Jul 24 14:56:31.431834 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.431813 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" event={"ID":"447552f9-3427-4e02-9ad1-fa205e53644f","Type":"ContainerStarted","Data":"fec109f9d478c385a2c0323846a3209e6acf76b8c12893209c03fc482f744f8c"} Jul 24 14:56:31.432981 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.432961 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-62nf6" event={"ID":"fde578a4-ae0f-425f-991c-30917584906f","Type":"ContainerStarted","Data":"136825be2ffb9eb4e6db7aed418f255abb339c2c5c94f0eceaf866be041bd752"} Jul 24 14:56:31.434330 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.434310 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ff2nx" event={"ID":"96ce8f23-8457-4b01-bde6-4caf871479b2","Type":"ContainerStarted","Data":"9339d7106e18dabf7a7ff719697cc37005ed1df151c340fcdab8f7ae69517998"} Jul 24 14:56:31.457986 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.457948 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pbnwh" podStartSLOduration=6.582388313 podStartE2EDuration="22.457933506s" totalImagesPullingTime="15.875545193s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:56:31.457452564 +0000 UTC m=+23.687929248" watchObservedRunningTime="2026-07-24 14:56:31.457933506 +0000 UTC m=+23.688410189" Jul 24 14:56:31.472931 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.472860 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hcxz7" podStartSLOduration=2.434640981 podStartE2EDuration="22.472842677s" totalImagesPullingTime="20.038201696s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:56:31.47251272 +0000 UTC m=+23.702989405" watchObservedRunningTime="2026-07-24 14:56:31.472842677 +0000 UTC m=+23.703319361" Jul 24 14:56:31.491416 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.491343 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ff2nx" podStartSLOduration=1.5152337550000001 podStartE2EDuration="22.491324641s" totalImagesPullingTime="20.976090886s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:56:31.490858961 +0000 UTC m=+23.721335646" watchObservedRunningTime="2026-07-24 14:56:31.491324641 +0000 UTC m=+23.721801326" Jul 24 14:56:31.505517 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.505481 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fcmhs" podStartSLOduration=2.449961148 podStartE2EDuration="22.505468751s" totalImagesPullingTime="20.055507603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:56:31.505087375 +0000 UTC m=+23.735564061" watchObservedRunningTime="2026-07-24 14:56:31.505468751 +0000 UTC m=+23.735945433" Jul 24 14:56:31.533276 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:31.533234 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal" podStartSLOduration=21.533221242 podStartE2EDuration="21.533221242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:56:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:56:31.532901524 +0000 UTC m=+23.763378226" watchObservedRunningTime="2026-07-24 14:56:31.533221242 +0000 UTC m=+23.763697925" Jul 24 14:56:32.132293 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:32.132124 2531 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 24 14:56:32.203526 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:32.203417 2531 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T14:56:32.132288634Z","UUID":"e8794fca-13a2-4da1-86c8-0a898d5175f5","Handler":null,"Name":"","Endpoint":""} Jul 24 14:56:32.204999 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:32.204976 2531 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 24 14:56:32.205141 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:32.205019 2531 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 24 14:56:32.271069 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:32.271043 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:32.271069 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:32.271063 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:32.271293 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:32.271160 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mjqx9" podUID="ec175d02-f2a5-4492-8133-e642a367da32" Jul 24 14:56:32.271357 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:32.271294 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xgswn" podUID="a9494b1d-f2bb-408b-8283-7d622ea5e450" Jul 24 14:56:32.438430 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:32.438398 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" event={"ID":"c4c7fe7d-701d-40f5-aafb-7608ab866182","Type":"ContainerStarted","Data":"bde17bd394b49fe7167486eff9b104dd57159270ffd259c5ba8c8ceb8ef5f4cd"} Jul 24 14:56:32.440148 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:32.440117 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s4h5d" event={"ID":"9c1cadc9-4ba8-451f-b647-22451763a4cc","Type":"ContainerStarted","Data":"9babd08a274582c418997e902008862cafd0dd98a72df2f31ff23c3b20b3ff35"} Jul 24 14:56:32.442093 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:32.442065 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" event={"ID":"d99284ebfc3675b576b1ca8cc30be861","Type":"ContainerStarted","Data":"846c11d50e280078372829ce413547ad0036fce411f6ec99d879fbecdd3cc978"} Jul 24 14:56:32.456284 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:32.456226 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-s4h5d" podStartSLOduration=2.566062998 podStartE2EDuration="23.456207976s" totalImagesPullingTime="20.890144978s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:56:32.456164731 +0000 UTC m=+24.686641425" watchObservedRunningTime="2026-07-24 14:56:32.456207976 +0000 UTC m=+24.686684662" Jul 24 14:56:32.456436 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:32.456409 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-62nf6" podStartSLOduration=2.576447137 podStartE2EDuration="23.456398867s" totalImagesPullingTime="20.87995173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:56:31.554043377 +0000 UTC m=+23.784520061" watchObservedRunningTime="2026-07-24 14:56:32.456398867 +0000 UTC m=+24.686875553" Jul 24 14:56:32.473619 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:32.473572 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" podStartSLOduration=22.473559632 podStartE2EDuration="22.473559632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:56:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:56:32.473251315 +0000 UTC m=+24.703728000" watchObservedRunningTime="2026-07-24 14:56:32.473559632 +0000 UTC m=+24.704036317" Jul 24 14:56:33.270917 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:33.270860 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:33.271070 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:33.271019 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jf8n" podUID="441a2a3d-5df6-4a88-92cc-63d18ba4310f" Jul 24 14:56:33.450013 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:33.449974 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" event={"ID":"447552f9-3427-4e02-9ad1-fa205e53644f","Type":"ContainerStarted","Data":"c485d649e941f3230888ed70b8cb0dd3f91c7453cd2781cd48d53ac5a33dec9e"} Jul 24 14:56:33.894629 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:33.894555 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fcmhs" Jul 24 14:56:33.895201 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:33.895174 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fcmhs" Jul 24 14:56:34.271129 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:34.271100 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:34.271322 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:34.271108 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:34.271322 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:34.271223 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mjqx9" podUID="ec175d02-f2a5-4492-8133-e642a367da32" Jul 24 14:56:34.271488 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:34.271323 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xgswn" podUID="a9494b1d-f2bb-408b-8283-7d622ea5e450" Jul 24 14:56:34.452930 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:34.452900 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fcmhs" Jul 24 14:56:34.453401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:34.453382 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fcmhs" Jul 24 14:56:35.271630 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:35.271601 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:35.271830 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:35.271707 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jf8n" podUID="441a2a3d-5df6-4a88-92cc-63d18ba4310f" Jul 24 14:56:36.271466 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:36.271302 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:36.271992 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:36.271376 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:36.271992 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:36.271553 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mjqx9" podUID="ec175d02-f2a5-4492-8133-e642a367da32" Jul 24 14:56:36.271992 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:36.271614 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xgswn" podUID="a9494b1d-f2bb-408b-8283-7d622ea5e450" Jul 24 14:56:36.458125 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:36.458099 2531 generic.go:356] "Generic (PLEG): container finished" podID="02657d23-d776-4da0-939a-e1f8893c5904" containerID="e8543f900607682e0bd71b50a7622e63f9525745e4f75eabc04954a1b70ae9a1" exitCode=0 Jul 24 14:56:36.458311 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:36.458176 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vgqjg" event={"ID":"02657d23-d776-4da0-939a-e1f8893c5904","Type":"ContainerDied","Data":"e8543f900607682e0bd71b50a7622e63f9525745e4f75eabc04954a1b70ae9a1"} Jul 24 14:56:36.460332 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:36.460304 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" event={"ID":"c4c7fe7d-701d-40f5-aafb-7608ab866182","Type":"ContainerStarted","Data":"40ad9ca72bcaaf1ba3b38963e0a127afb4945acd1aa05a35d5e8b97cd5ae6412"} Jul 24 14:56:36.463895 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:36.463852 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" event={"ID":"447552f9-3427-4e02-9ad1-fa205e53644f","Type":"ContainerStarted","Data":"996e8910ebb80416320cd053b995ded1363a19a7766516581358bc5130da957d"} Jul 24 14:56:36.513490 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:36.513448 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" podStartSLOduration=6.57255063 podStartE2EDuration="27.513434134s" totalImagesPullingTime="20.940883504s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:56:36.512355704 +0000 UTC m=+28.742832411" watchObservedRunningTime="2026-07-24 14:56:36.513434134 +0000 UTC m=+28.743910817" Jul 24 14:56:36.537354 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:36.537309 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bsxq" podStartSLOduration=2.791565106 podStartE2EDuration="27.53729696s" totalImagesPullingTime="24.745731854s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:56:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:56:36.537196251 +0000 UTC m=+28.767672935" watchObservedRunningTime="2026-07-24 14:56:36.53729696 +0000 UTC m=+28.767773643" Jul 24 14:56:37.271484 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:37.271450 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:37.272042 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:37.271584 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jf8n" podUID="441a2a3d-5df6-4a88-92cc-63d18ba4310f" Jul 24 14:56:37.466989 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:37.466956 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:37.467155 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:37.467038 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:37.467155 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:37.467116 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:37.482757 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:37.482730 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:37.482987 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:37.482975 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:56:37.903178 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:37.902928 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mjqx9"] Jul 24 14:56:37.903327 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:37.903210 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:37.903327 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:37.903310 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mjqx9" podUID="ec175d02-f2a5-4492-8133-e642a367da32" Jul 24 14:56:37.903563 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:37.903543 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6jf8n"] Jul 24 14:56:37.903635 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:37.903622 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:37.903717 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:37.903697 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jf8n" podUID="441a2a3d-5df6-4a88-92cc-63d18ba4310f" Jul 24 14:56:37.907882 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:37.907844 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xgswn"] Jul 24 14:56:37.907989 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:37.907968 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:37.908093 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:37.908072 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xgswn" podUID="a9494b1d-f2bb-408b-8283-7d622ea5e450" Jul 24 14:56:38.470155 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:38.470122 2531 generic.go:356] "Generic (PLEG): container finished" podID="02657d23-d776-4da0-939a-e1f8893c5904" containerID="942784726dede2455af6166584da7c6c37e1bdda0005c779fc01ba4ef860193d" exitCode=0 Jul 24 14:56:38.470658 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:38.470197 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vgqjg" event={"ID":"02657d23-d776-4da0-939a-e1f8893c5904","Type":"ContainerDied","Data":"942784726dede2455af6166584da7c6c37e1bdda0005c779fc01ba4ef860193d"} Jul 24 14:56:39.271269 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:39.271225 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:39.271423 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:39.271225 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:39.271423 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:39.271344 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xgswn" podUID="a9494b1d-f2bb-408b-8283-7d622ea5e450" Jul 24 14:56:39.271509 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:39.271422 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mjqx9" podUID="ec175d02-f2a5-4492-8133-e642a367da32" Jul 24 14:56:40.271577 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:40.271548 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:40.271944 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:40.271711 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jf8n" podUID="441a2a3d-5df6-4a88-92cc-63d18ba4310f" Jul 24 14:56:40.477305 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:40.477272 2531 generic.go:356] "Generic (PLEG): container finished" podID="02657d23-d776-4da0-939a-e1f8893c5904" containerID="fb33ece03b343eefa7f1111117ecc87309633020a9210ab198b2d48f22bf6b4c" exitCode=0 Jul 24 14:56:40.477453 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:40.477342 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vgqjg" event={"ID":"02657d23-d776-4da0-939a-e1f8893c5904","Type":"ContainerDied","Data":"fb33ece03b343eefa7f1111117ecc87309633020a9210ab198b2d48f22bf6b4c"} Jul 24 14:56:40.893687 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:40.893652 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs\") pod \"network-metrics-daemon-xgswn\" (UID: \"a9494b1d-f2bb-408b-8283-7d622ea5e450\") " pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:40.893890 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:40.893743 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4rz7\" (UniqueName: \"kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7\") pod \"network-check-target-mjqx9\" (UID: \"ec175d02-f2a5-4492-8133-e642a367da32\") " pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:40.893890 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:40.893804 2531 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 14:56:40.893890 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:40.893857 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 14:56:40.894048 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:40.893898 2531 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 14:56:40.894048 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:40.893911 2531 projected.go:196] Error preparing data for projected volume kube-api-access-z4rz7 for pod openshift-network-diagnostics/network-check-target-mjqx9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 14:56:40.894048 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:40.893881 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs podName:a9494b1d-f2bb-408b-8283-7d622ea5e450 nodeName:}" failed. No retries permitted until 2026-07-24 14:57:12.893847937 +0000 UTC m=+65.124324602 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs") pod "network-metrics-daemon-xgswn" (UID: "a9494b1d-f2bb-408b-8283-7d622ea5e450") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 14:56:40.894048 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:40.893971 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7 podName:ec175d02-f2a5-4492-8133-e642a367da32 nodeName:}" failed. No retries permitted until 2026-07-24 14:57:12.893955308 +0000 UTC m=+65.124431976 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-z4rz7" (UniqueName: "kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7") pod "network-check-target-mjqx9" (UID: "ec175d02-f2a5-4492-8133-e642a367da32") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 14:56:40.994038 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:40.994008 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret\") pod \"global-pull-secret-syncer-6jf8n\" (UID: \"441a2a3d-5df6-4a88-92cc-63d18ba4310f\") " pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:40.994154 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:40.994141 2531 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 14:56:40.994216 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:40.994205 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret podName:441a2a3d-5df6-4a88-92cc-63d18ba4310f nodeName:}" failed. No retries permitted until 2026-07-24 14:57:12.994188144 +0000 UTC m=+65.224664806 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret") pod "global-pull-secret-syncer-6jf8n" (UID: "441a2a3d-5df6-4a88-92cc-63d18ba4310f") : object "kube-system"/"original-pull-secret" not registered Jul 24 14:56:41.271378 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:41.271356 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:41.271520 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:41.271356 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:41.271520 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:41.271451 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xgswn" podUID="a9494b1d-f2bb-408b-8283-7d622ea5e450" Jul 24 14:56:41.271589 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:41.271521 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mjqx9" podUID="ec175d02-f2a5-4492-8133-e642a367da32" Jul 24 14:56:42.271556 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:42.271523 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:42.272146 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:42.271637 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6jf8n" podUID="441a2a3d-5df6-4a88-92cc-63d18ba4310f" Jul 24 14:56:43.060999 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.060973 2531 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeReady" Jul 24 14:56:43.061161 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.061115 2531 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 24 14:56:43.122948 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.122911 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-776fc686df-t8g55"] Jul 24 14:56:43.125532 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.125507 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:43.130410 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.130382 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qkp5x"] Jul 24 14:56:43.136491 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.133285 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-779758b8b8-dc8j5"] Jul 24 14:56:43.136491 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.133859 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 24 14:56:43.136491 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.133982 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 24 14:56:43.136491 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.134320 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 24 14:56:43.136491 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.135180 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 24 14:56:43.136491 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.135326 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-26tts\"" Jul 24 14:56:43.136491 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.135783 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 24 14:56:43.136491 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.136105 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qkp5x" Jul 24 14:56:43.137541 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.137206 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 24 14:56:43.137761 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.137737 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ndjm9"] Jul 24 14:56:43.138265 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.138233 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.139082 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.139062 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-2r2hf\"" Jul 24 14:56:43.139631 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.139612 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-77k6k"] Jul 24 14:56:43.139919 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.139900 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ndjm9" Jul 24 14:56:43.141473 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.141455 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525"] Jul 24 14:56:43.141614 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.141590 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" Jul 24 14:56:43.143442 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.143422 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn"] Jul 24 14:56:43.143547 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.143528 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525" Jul 24 14:56:43.144585 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.144568 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 24 14:56:43.144664 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.144651 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 24 14:56:43.145401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.145383 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn" Jul 24 14:56:43.150679 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.150651 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 24 14:56:43.150776 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.150753 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 24 14:56:43.151090 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.150783 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 24 14:56:43.151193 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.151114 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 24 14:56:43.151193 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.150978 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 24 14:56:43.151314 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.151006 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 24 14:56:43.151314 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.151064 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 24 14:56:43.151414 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.151054 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 24 14:56:43.151414 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.151362 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 24 14:56:43.151512 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.151489 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 24 14:56:43.151715 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.151692 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 24 14:56:43.151893 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.151850 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 24 14:56:43.152063 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.152034 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 24 14:56:43.152186 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.152095 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 24 14:56:43.152300 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.152282 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 24 14:56:43.152402 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.152311 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 24 14:56:43.153134 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.153113 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-z9b7f\"" Jul 24 14:56:43.153396 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.153381 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 24 14:56:43.153486 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.153415 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-776fc686df-t8g55"] Jul 24 14:56:43.153638 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.153622 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tq8dm\"" Jul 24 14:56:43.153884 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.153843 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 24 14:56:43.154011 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.153995 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-zbdbq\"" Jul 24 14:56:43.154845 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.154828 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-czcf5\"" Jul 24 14:56:43.156040 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.156018 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qkp5x"] Jul 24 14:56:43.156897 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.156857 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9bsmn\"" Jul 24 14:56:43.171185 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.170143 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-wjxvg"] Jul 24 14:56:43.171704 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.171680 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 24 14:56:43.172565 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.172545 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-dntl4"] Jul 24 14:56:43.172688 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.172670 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-wjxvg" Jul 24 14:56:43.177063 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.176481 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 24 14:56:43.177063 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.176799 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 24 14:56:43.177063 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.177036 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 24 14:56:43.179820 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.179792 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 24 14:56:43.180475 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.180452 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 24 14:56:43.180721 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.180705 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ndjm9"] Jul 24 14:56:43.180815 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.180741 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-nt4bv"] Jul 24 14:56:43.182493 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.182170 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-5gtjq\"" Jul 24 14:56:43.184111 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.184090 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-dntl4" Jul 24 14:56:43.187765 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.187611 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 24 14:56:43.191449 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.191429 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 24 14:56:43.192163 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.191963 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2jdtw"] Jul 24 14:56:43.192163 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.192002 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-v2kd8\"" Jul 24 14:56:43.192944 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.192702 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 24 14:56:43.193157 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.193141 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nt4bv" Jul 24 14:56:43.194285 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.194232 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc"] Jul 24 14:56:43.194480 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.194461 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2jdtw" Jul 24 14:56:43.195406 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.195390 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 14:56:43.196061 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.196043 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525"] Jul 24 14:56:43.196209 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.196193 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-s4b7h"] Jul 24 14:56:43.196333 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.196308 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc" Jul 24 14:56:43.197428 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.197410 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 24 14:56:43.197654 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.197624 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 14:56:43.197787 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.197625 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-6qptn\"" Jul 24 14:56:43.197854 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.197790 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 24 14:56:43.198483 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.198463 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-779758b8b8-dc8j5"] Jul 24 14:56:43.198572 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.198502 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-77k6k"] Jul 24 14:56:43.198572 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.198521 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn"] Jul 24 14:56:43.198572 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.198539 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-dntl4"] Jul 24 14:56:43.198572 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.198556 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc"] Jul 24 14:56:43.198572 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.198573 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s4b7h"] Jul 24 14:56:43.198780 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.198589 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2jdtw"] Jul 24 14:56:43.198780 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.198643 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s4b7h" Jul 24 14:56:43.199482 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.199466 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-wjxvg"] Jul 24 14:56:43.200166 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.200149 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 24 14:56:43.200281 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.200267 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 24 14:56:43.200361 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.200264 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-zh6r4\"" Jul 24 14:56:43.200361 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.200313 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 24 14:56:43.200469 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.200270 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bc4wp\"" Jul 24 14:56:43.200469 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.200200 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 24 14:56:43.201232 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.201201 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 24 14:56:43.201349 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.201249 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 24 14:56:43.201349 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.201205 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 24 14:56:43.201487 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.201472 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-55njp\"" Jul 24 14:56:43.208413 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.208395 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-nt4bv"] Jul 24 14:56:43.211257 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.211237 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/520066e2-8b42-4872-b257-20df38f77e37-config\") pod \"console-operator-584dcb6b54-77k6k\" (UID: \"520066e2-8b42-4872-b257-20df38f77e37\") " pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" Jul 24 14:56:43.211351 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.211268 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/888b99fe-7dd7-499b-9326-37b892a7ef56-image-registry-private-configuration\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.211351 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.211292 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.211351 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.211340 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn7cq\" (UniqueName: \"kubernetes.io/projected/72091411-787f-478a-a97f-8ebc53464929-kube-api-access-gn7cq\") pod \"volume-data-source-validator-549c6d54cf-ndjm9\" (UID: \"72091411-787f-478a-a97f-8ebc53464929\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ndjm9" Jul 24 14:56:43.211510 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.211385 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/520066e2-8b42-4872-b257-20df38f77e37-serving-cert\") pod \"console-operator-584dcb6b54-77k6k\" (UID: \"520066e2-8b42-4872-b257-20df38f77e37\") " pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" Jul 24 14:56:43.211510 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.211422 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:43.211510 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.211457 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-bound-sa-token\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.211658 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.211525 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvkqc\" (UniqueName: \"kubernetes.io/projected/520066e2-8b42-4872-b257-20df38f77e37-kube-api-access-rvkqc\") pod \"console-operator-584dcb6b54-77k6k\" (UID: \"520066e2-8b42-4872-b257-20df38f77e37\") " pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" Jul 24 14:56:43.211658 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.211550 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-default-certificate\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:43.211658 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.211576 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/888b99fe-7dd7-499b-9326-37b892a7ef56-ca-trust-extracted\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.211658 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.211605 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-stats-auth\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:43.211658 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.211629 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/079183d0-ba31-4d48-aa8f-e11069836062-serving-cert\") pod \"service-ca-operator-5d6c75f84f-qkp5x\" (UID: \"079183d0-ba31-4d48-aa8f-e11069836062\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qkp5x" Jul 24 14:56:43.211883 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.211677 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-certificates\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.211883 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.211701 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/888b99fe-7dd7-499b-9326-37b892a7ef56-trusted-ca\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.211883 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.211724 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v559v\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-kube-api-access-v559v\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.211883 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.211804 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfsnh\" (UniqueName: \"kubernetes.io/projected/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-kube-api-access-hfsnh\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:43.211883 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.211834 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dpvw\" (UniqueName: \"kubernetes.io/projected/079183d0-ba31-4d48-aa8f-e11069836062-kube-api-access-6dpvw\") pod \"service-ca-operator-5d6c75f84f-qkp5x\" (UID: \"079183d0-ba31-4d48-aa8f-e11069836062\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qkp5x" Jul 24 14:56:43.212120 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.211903 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:43.212120 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.211927 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/079183d0-ba31-4d48-aa8f-e11069836062-config\") pod \"service-ca-operator-5d6c75f84f-qkp5x\" (UID: \"079183d0-ba31-4d48-aa8f-e11069836062\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qkp5x" Jul 24 14:56:43.212120 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.211955 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/520066e2-8b42-4872-b257-20df38f77e37-trusted-ca\") pod \"console-operator-584dcb6b54-77k6k\" (UID: \"520066e2-8b42-4872-b257-20df38f77e37\") " pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" Jul 24 14:56:43.212392 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.212340 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/888b99fe-7dd7-499b-9326-37b892a7ef56-installation-pull-secrets\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.271485 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.271463 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:56:43.271677 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.271659 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:56:43.274825 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.274811 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 14:56:43.275279 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.275265 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j76t9\"" Jul 24 14:56:43.275370 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.275354 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5tbx2\"" Jul 24 14:56:43.312671 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.312645 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v559v\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-kube-api-access-v559v\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.312784 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.312689 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/595851c1-64b8-44ad-b819-f6b635101e2e-tmp\") pod \"insights-operator-5885fdb759-wjxvg\" (UID: \"595851c1-64b8-44ad-b819-f6b635101e2e\") " pod="openshift-insights/insights-operator-5885fdb759-wjxvg" Jul 24 14:56:43.312784 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.312717 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/595851c1-64b8-44ad-b819-f6b635101e2e-service-ca-bundle\") pod \"insights-operator-5885fdb759-wjxvg\" (UID: \"595851c1-64b8-44ad-b819-f6b635101e2e\") " pod="openshift-insights/insights-operator-5885fdb759-wjxvg" Jul 24 14:56:43.312943 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.312926 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dntl4\" (UID: \"020f32d2-c323-430c-b6a6-d97a25182801\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dntl4" Jul 24 14:56:43.313010 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.312957 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vl525\" (UID: \"afe11285-00c7-47a8-a0d0-08acd3df5bda\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525" Jul 24 14:56:43.313010 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.312987 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54j49\" (UniqueName: \"kubernetes.io/projected/595851c1-64b8-44ad-b819-f6b635101e2e-kube-api-access-54j49\") pod \"insights-operator-5885fdb759-wjxvg\" (UID: \"595851c1-64b8-44ad-b819-f6b635101e2e\") " pod="openshift-insights/insights-operator-5885fdb759-wjxvg" Jul 24 14:56:43.313111 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313019 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:43.313111 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313045 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/079183d0-ba31-4d48-aa8f-e11069836062-config\") pod \"service-ca-operator-5d6c75f84f-qkp5x\" (UID: \"079183d0-ba31-4d48-aa8f-e11069836062\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qkp5x" Jul 24 14:56:43.313111 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313071 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlsgj\" (UniqueName: \"kubernetes.io/projected/afe11285-00c7-47a8-a0d0-08acd3df5bda-kube-api-access-zlsgj\") pod \"cluster-samples-operator-6748489656-vl525\" (UID: \"afe11285-00c7-47a8-a0d0-08acd3df5bda\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525" Jul 24 14:56:43.313111 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313102 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/888b99fe-7dd7-499b-9326-37b892a7ef56-installation-pull-secrets\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.313303 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313125 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e5d1043-9361-463b-a342-bcb81a9f181d-cert\") pod \"ingress-canary-s4b7h\" (UID: \"9e5d1043-9361-463b-a342-bcb81a9f181d\") " pod="openshift-ingress-canary/ingress-canary-s4b7h" Jul 24 14:56:43.313303 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313155 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/520066e2-8b42-4872-b257-20df38f77e37-config\") pod \"console-operator-584dcb6b54-77k6k\" (UID: \"520066e2-8b42-4872-b257-20df38f77e37\") " pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" Jul 24 14:56:43.313303 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313184 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/888b99fe-7dd7-499b-9326-37b892a7ef56-image-registry-private-configuration\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.313303 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313212 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mw7p\" (UniqueName: \"kubernetes.io/projected/9e5d1043-9361-463b-a342-bcb81a9f181d-kube-api-access-9mw7p\") pod \"ingress-canary-s4b7h\" (UID: \"9e5d1043-9361-463b-a342-bcb81a9f181d\") " pod="openshift-ingress-canary/ingress-canary-s4b7h" Jul 24 14:56:43.313303 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313237 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31f123ed-b090-4d80-a46b-888032edc051-tmp-dir\") pod \"dns-default-2jdtw\" (UID: \"31f123ed-b090-4d80-a46b-888032edc051\") " pod="openshift-dns/dns-default-2jdtw" Jul 24 14:56:43.313303 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313268 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/520066e2-8b42-4872-b257-20df38f77e37-serving-cert\") pod \"console-operator-584dcb6b54-77k6k\" (UID: \"520066e2-8b42-4872-b257-20df38f77e37\") " pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" Jul 24 14:56:43.313303 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313295 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls\") pod \"dns-default-2jdtw\" (UID: \"31f123ed-b090-4d80-a46b-888032edc051\") " pod="openshift-dns/dns-default-2jdtw" Jul 24 14:56:43.313620 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313318 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/31f123ed-b090-4d80-a46b-888032edc051-config-volume\") pod \"dns-default-2jdtw\" (UID: \"31f123ed-b090-4d80-a46b-888032edc051\") " pod="openshift-dns/dns-default-2jdtw" Jul 24 14:56:43.313620 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313345 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/91fda333-13a3-4954-8c20-e3c8c8bd4da3-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc\" (UID: \"91fda333-13a3-4954-8c20-e3c8c8bd4da3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc" Jul 24 14:56:43.313620 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313369 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j22g\" (UniqueName: \"kubernetes.io/projected/40eda5e0-b362-4b86-9ebe-87f141f0fee9-kube-api-access-9j22g\") pod \"network-check-source-7d88cb55d8-nt4bv\" (UID: \"40eda5e0-b362-4b86-9ebe-87f141f0fee9\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nt4bv" Jul 24 14:56:43.313620 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313400 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvkqc\" (UniqueName: \"kubernetes.io/projected/520066e2-8b42-4872-b257-20df38f77e37-kube-api-access-rvkqc\") pod \"console-operator-584dcb6b54-77k6k\" (UID: \"520066e2-8b42-4872-b257-20df38f77e37\") " pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" Jul 24 14:56:43.313620 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313424 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-default-certificate\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:43.313620 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313453 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-stats-auth\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:43.313620 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313481 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/079183d0-ba31-4d48-aa8f-e11069836062-serving-cert\") pod \"service-ca-operator-5d6c75f84f-qkp5x\" (UID: \"079183d0-ba31-4d48-aa8f-e11069836062\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qkp5x" Jul 24 14:56:43.313620 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313513 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/888b99fe-7dd7-499b-9326-37b892a7ef56-trusted-ca\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.313620 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313523 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/079183d0-ba31-4d48-aa8f-e11069836062-config\") pod \"service-ca-operator-5d6c75f84f-qkp5x\" (UID: \"079183d0-ba31-4d48-aa8f-e11069836062\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qkp5x" Jul 24 14:56:43.313620 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313539 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-certificates\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.313620 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313569 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/595851c1-64b8-44ad-b819-f6b635101e2e-serving-cert\") pod \"insights-operator-5885fdb759-wjxvg\" (UID: \"595851c1-64b8-44ad-b819-f6b635101e2e\") " pod="openshift-insights/insights-operator-5885fdb759-wjxvg" Jul 24 14:56:43.313620 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313593 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfsnh\" (UniqueName: \"kubernetes.io/projected/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-kube-api-access-hfsnh\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:43.313620 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313612 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dpvw\" (UniqueName: \"kubernetes.io/projected/079183d0-ba31-4d48-aa8f-e11069836062-kube-api-access-6dpvw\") pod \"service-ca-operator-5d6c75f84f-qkp5x\" (UID: \"079183d0-ba31-4d48-aa8f-e11069836062\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qkp5x" Jul 24 14:56:43.314297 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313631 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/595851c1-64b8-44ad-b819-f6b635101e2e-snapshots\") pod \"insights-operator-5885fdb759-wjxvg\" (UID: \"595851c1-64b8-44ad-b819-f6b635101e2e\") " pod="openshift-insights/insights-operator-5885fdb759-wjxvg" Jul 24 14:56:43.314297 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313654 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-bound-sa-token\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.314297 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313671 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-86bxn\" (UID: \"2872026a-6cfb-4c62-8ae5-b33297a9b1e7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn" Jul 24 14:56:43.314297 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313707 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.314297 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313738 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsc6q\" (UniqueName: \"kubernetes.io/projected/91fda333-13a3-4954-8c20-e3c8c8bd4da3-kube-api-access-gsc6q\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc\" (UID: \"91fda333-13a3-4954-8c20-e3c8c8bd4da3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc" Jul 24 14:56:43.314297 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313762 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlb4r\" (UniqueName: \"kubernetes.io/projected/31f123ed-b090-4d80-a46b-888032edc051-kube-api-access-qlb4r\") pod \"dns-default-2jdtw\" (UID: \"31f123ed-b090-4d80-a46b-888032edc051\") " pod="openshift-dns/dns-default-2jdtw" Jul 24 14:56:43.314297 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313791 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn7cq\" (UniqueName: \"kubernetes.io/projected/72091411-787f-478a-a97f-8ebc53464929-kube-api-access-gn7cq\") pod \"volume-data-source-validator-549c6d54cf-ndjm9\" (UID: \"72091411-787f-478a-a97f-8ebc53464929\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ndjm9" Jul 24 14:56:43.314297 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313831 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-86bxn\" (UID: \"2872026a-6cfb-4c62-8ae5-b33297a9b1e7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn" Jul 24 14:56:43.314297 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313850 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:43.314297 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.313909 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/595851c1-64b8-44ad-b819-f6b635101e2e-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-wjxvg\" (UID: \"595851c1-64b8-44ad-b819-f6b635101e2e\") " pod="openshift-insights/insights-operator-5885fdb759-wjxvg" Jul 24 14:56:43.314297 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.314112 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/888b99fe-7dd7-499b-9326-37b892a7ef56-ca-trust-extracted\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.314297 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.314123 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-certificates\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.314297 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.314174 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/020f32d2-c323-430c-b6a6-d97a25182801-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-dntl4\" (UID: \"020f32d2-c323-430c-b6a6-d97a25182801\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dntl4" Jul 24 14:56:43.314297 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.314258 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlmtd\" (UniqueName: \"kubernetes.io/projected/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-kube-api-access-hlmtd\") pod \"cluster-monitoring-operator-548b489cb8-86bxn\" (UID: \"2872026a-6cfb-4c62-8ae5-b33297a9b1e7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn" Jul 24 14:56:43.314946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.314463 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/91fda333-13a3-4954-8c20-e3c8c8bd4da3-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc\" (UID: \"91fda333-13a3-4954-8c20-e3c8c8bd4da3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc" Jul 24 14:56:43.314946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.314494 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/520066e2-8b42-4872-b257-20df38f77e37-trusted-ca\") pod \"console-operator-584dcb6b54-77k6k\" (UID: \"520066e2-8b42-4872-b257-20df38f77e37\") " pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" Jul 24 14:56:43.314946 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.314548 2531 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 14:56:43.314946 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.314564 2531 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-779758b8b8-dc8j5: secret "image-registry-tls" not found Jul 24 14:56:43.314946 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.314622 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls podName:888b99fe-7dd7-499b-9326-37b892a7ef56 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:43.814604872 +0000 UTC m=+36.045081550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls") pod "image-registry-779758b8b8-dc8j5" (UID: "888b99fe-7dd7-499b-9326-37b892a7ef56") : secret "image-registry-tls" not found Jul 24 14:56:43.314946 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.314882 2531 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 14:56:43.314946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.314921 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/888b99fe-7dd7-499b-9326-37b892a7ef56-ca-trust-extracted\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.315262 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.315054 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle podName:3ede17c9-5194-41b7-a4e6-55a258fe1a4e nodeName:}" failed. No retries permitted until 2026-07-24 14:56:43.81503493 +0000 UTC m=+36.045511614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle") pod "router-default-776fc686df-t8g55" (UID: "3ede17c9-5194-41b7-a4e6-55a258fe1a4e") : configmap references non-existent config key: service-ca.crt Jul 24 14:56:43.315262 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.315077 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs podName:3ede17c9-5194-41b7-a4e6-55a258fe1a4e nodeName:}" failed. No retries permitted until 2026-07-24 14:56:43.815066319 +0000 UTC m=+36.045542987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs") pod "router-default-776fc686df-t8g55" (UID: "3ede17c9-5194-41b7-a4e6-55a258fe1a4e") : secret "router-metrics-certs-default" not found Jul 24 14:56:43.318962 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.318738 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/520066e2-8b42-4872-b257-20df38f77e37-config\") pod \"console-operator-584dcb6b54-77k6k\" (UID: \"520066e2-8b42-4872-b257-20df38f77e37\") " pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" Jul 24 14:56:43.319897 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.318943 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/888b99fe-7dd7-499b-9326-37b892a7ef56-trusted-ca\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.319897 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.319766 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-default-certificate\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:43.326639 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.326616 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/520066e2-8b42-4872-b257-20df38f77e37-trusted-ca\") pod \"console-operator-584dcb6b54-77k6k\" (UID: \"520066e2-8b42-4872-b257-20df38f77e37\") " pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" Jul 24 14:56:43.328044 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.328022 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/520066e2-8b42-4872-b257-20df38f77e37-serving-cert\") pod \"console-operator-584dcb6b54-77k6k\" (UID: \"520066e2-8b42-4872-b257-20df38f77e37\") " pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" Jul 24 14:56:43.328565 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.328547 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-stats-auth\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:43.328679 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.328643 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/888b99fe-7dd7-499b-9326-37b892a7ef56-installation-pull-secrets\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.328679 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.328666 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/079183d0-ba31-4d48-aa8f-e11069836062-serving-cert\") pod \"service-ca-operator-5d6c75f84f-qkp5x\" (UID: \"079183d0-ba31-4d48-aa8f-e11069836062\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qkp5x" Jul 24 14:56:43.329455 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.329433 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/888b99fe-7dd7-499b-9326-37b892a7ef56-image-registry-private-configuration\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.333064 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.332041 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:43.334292 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.334268 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn7cq\" (UniqueName: \"kubernetes.io/projected/72091411-787f-478a-a97f-8ebc53464929-kube-api-access-gn7cq\") pod \"volume-data-source-validator-549c6d54cf-ndjm9\" (UID: \"72091411-787f-478a-a97f-8ebc53464929\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ndjm9" Jul 24 14:56:43.335025 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.334982 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dpvw\" (UniqueName: \"kubernetes.io/projected/079183d0-ba31-4d48-aa8f-e11069836062-kube-api-access-6dpvw\") pod \"service-ca-operator-5d6c75f84f-qkp5x\" (UID: \"079183d0-ba31-4d48-aa8f-e11069836062\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qkp5x" Jul 24 14:56:43.335481 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.335465 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v559v\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-kube-api-access-v559v\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.343546 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.343521 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-bound-sa-token\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.343657 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.343643 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvkqc\" (UniqueName: \"kubernetes.io/projected/520066e2-8b42-4872-b257-20df38f77e37-kube-api-access-rvkqc\") pod \"console-operator-584dcb6b54-77k6k\" (UID: \"520066e2-8b42-4872-b257-20df38f77e37\") " pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" Jul 24 14:56:43.344770 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.344744 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfsnh\" (UniqueName: \"kubernetes.io/projected/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-kube-api-access-hfsnh\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:43.414991 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.414959 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/595851c1-64b8-44ad-b819-f6b635101e2e-serving-cert\") pod \"insights-operator-5885fdb759-wjxvg\" (UID: \"595851c1-64b8-44ad-b819-f6b635101e2e\") " pod="openshift-insights/insights-operator-5885fdb759-wjxvg" Jul 24 14:56:43.415148 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.415004 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/595851c1-64b8-44ad-b819-f6b635101e2e-snapshots\") pod \"insights-operator-5885fdb759-wjxvg\" (UID: \"595851c1-64b8-44ad-b819-f6b635101e2e\") " pod="openshift-insights/insights-operator-5885fdb759-wjxvg" Jul 24 14:56:43.415148 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.415024 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-86bxn\" (UID: \"2872026a-6cfb-4c62-8ae5-b33297a9b1e7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn" Jul 24 14:56:43.415148 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.415051 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsc6q\" (UniqueName: \"kubernetes.io/projected/91fda333-13a3-4954-8c20-e3c8c8bd4da3-kube-api-access-gsc6q\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc\" (UID: \"91fda333-13a3-4954-8c20-e3c8c8bd4da3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc" Jul 24 14:56:43.415148 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.415076 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlb4r\" (UniqueName: \"kubernetes.io/projected/31f123ed-b090-4d80-a46b-888032edc051-kube-api-access-qlb4r\") pod \"dns-default-2jdtw\" (UID: \"31f123ed-b090-4d80-a46b-888032edc051\") " pod="openshift-dns/dns-default-2jdtw" Jul 24 14:56:43.415148 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.415105 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-86bxn\" (UID: \"2872026a-6cfb-4c62-8ae5-b33297a9b1e7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn" Jul 24 14:56:43.415390 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.415168 2531 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 14:56:43.415390 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.415219 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls podName:2872026a-6cfb-4c62-8ae5-b33297a9b1e7 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:43.915206055 +0000 UTC m=+36.145682716 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-86bxn" (UID: "2872026a-6cfb-4c62-8ae5-b33297a9b1e7") : secret "cluster-monitoring-operator-tls" not found Jul 24 14:56:43.415567 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.415548 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/595851c1-64b8-44ad-b819-f6b635101e2e-snapshots\") pod \"insights-operator-5885fdb759-wjxvg\" (UID: \"595851c1-64b8-44ad-b819-f6b635101e2e\") " pod="openshift-insights/insights-operator-5885fdb759-wjxvg" Jul 24 14:56:43.415643 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.415597 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/595851c1-64b8-44ad-b819-f6b635101e2e-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-wjxvg\" (UID: \"595851c1-64b8-44ad-b819-f6b635101e2e\") " pod="openshift-insights/insights-operator-5885fdb759-wjxvg" Jul 24 14:56:43.415643 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.415628 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/020f32d2-c323-430c-b6a6-d97a25182801-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-dntl4\" (UID: \"020f32d2-c323-430c-b6a6-d97a25182801\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dntl4" Jul 24 14:56:43.415747 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.415649 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlmtd\" (UniqueName: \"kubernetes.io/projected/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-kube-api-access-hlmtd\") pod \"cluster-monitoring-operator-548b489cb8-86bxn\" (UID: \"2872026a-6cfb-4c62-8ae5-b33297a9b1e7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn" Jul 24 14:56:43.415747 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.415676 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/91fda333-13a3-4954-8c20-e3c8c8bd4da3-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc\" (UID: \"91fda333-13a3-4954-8c20-e3c8c8bd4da3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc" Jul 24 14:56:43.415934 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.415911 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-86bxn\" (UID: \"2872026a-6cfb-4c62-8ae5-b33297a9b1e7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn" Jul 24 14:56:43.416226 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.416181 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/595851c1-64b8-44ad-b819-f6b635101e2e-tmp\") pod \"insights-operator-5885fdb759-wjxvg\" (UID: \"595851c1-64b8-44ad-b819-f6b635101e2e\") " pod="openshift-insights/insights-operator-5885fdb759-wjxvg" Jul 24 14:56:43.416356 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.416229 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/595851c1-64b8-44ad-b819-f6b635101e2e-service-ca-bundle\") pod \"insights-operator-5885fdb759-wjxvg\" (UID: \"595851c1-64b8-44ad-b819-f6b635101e2e\") " pod="openshift-insights/insights-operator-5885fdb759-wjxvg" Jul 24 14:56:43.416356 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.416276 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dntl4\" (UID: \"020f32d2-c323-430c-b6a6-d97a25182801\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dntl4" Jul 24 14:56:43.416356 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.416307 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vl525\" (UID: \"afe11285-00c7-47a8-a0d0-08acd3df5bda\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525" Jul 24 14:56:43.416356 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.416336 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/020f32d2-c323-430c-b6a6-d97a25182801-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-dntl4\" (UID: \"020f32d2-c323-430c-b6a6-d97a25182801\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dntl4" Jul 24 14:56:43.416562 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.416362 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54j49\" (UniqueName: \"kubernetes.io/projected/595851c1-64b8-44ad-b819-f6b635101e2e-kube-api-access-54j49\") pod \"insights-operator-5885fdb759-wjxvg\" (UID: \"595851c1-64b8-44ad-b819-f6b635101e2e\") " pod="openshift-insights/insights-operator-5885fdb759-wjxvg" Jul 24 14:56:43.416562 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.416400 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlsgj\" (UniqueName: \"kubernetes.io/projected/afe11285-00c7-47a8-a0d0-08acd3df5bda-kube-api-access-zlsgj\") pod \"cluster-samples-operator-6748489656-vl525\" (UID: \"afe11285-00c7-47a8-a0d0-08acd3df5bda\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525" Jul 24 14:56:43.416562 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.416427 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e5d1043-9361-463b-a342-bcb81a9f181d-cert\") pod \"ingress-canary-s4b7h\" (UID: \"9e5d1043-9361-463b-a342-bcb81a9f181d\") " pod="openshift-ingress-canary/ingress-canary-s4b7h" Jul 24 14:56:43.416562 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.416459 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mw7p\" (UniqueName: \"kubernetes.io/projected/9e5d1043-9361-463b-a342-bcb81a9f181d-kube-api-access-9mw7p\") pod \"ingress-canary-s4b7h\" (UID: \"9e5d1043-9361-463b-a342-bcb81a9f181d\") " pod="openshift-ingress-canary/ingress-canary-s4b7h" Jul 24 14:56:43.416562 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.416485 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31f123ed-b090-4d80-a46b-888032edc051-tmp-dir\") pod \"dns-default-2jdtw\" (UID: \"31f123ed-b090-4d80-a46b-888032edc051\") " pod="openshift-dns/dns-default-2jdtw" Jul 24 14:56:43.416562 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.416499 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/595851c1-64b8-44ad-b819-f6b635101e2e-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-wjxvg\" (UID: \"595851c1-64b8-44ad-b819-f6b635101e2e\") " pod="openshift-insights/insights-operator-5885fdb759-wjxvg" Jul 24 14:56:43.416562 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.416513 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls\") pod \"dns-default-2jdtw\" (UID: \"31f123ed-b090-4d80-a46b-888032edc051\") " pod="openshift-dns/dns-default-2jdtw" Jul 24 14:56:43.416562 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.416544 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/31f123ed-b090-4d80-a46b-888032edc051-config-volume\") pod \"dns-default-2jdtw\" (UID: \"31f123ed-b090-4d80-a46b-888032edc051\") " pod="openshift-dns/dns-default-2jdtw" Jul 24 14:56:43.416969 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.416570 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/91fda333-13a3-4954-8c20-e3c8c8bd4da3-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc\" (UID: \"91fda333-13a3-4954-8c20-e3c8c8bd4da3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc" Jul 24 14:56:43.416969 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.416580 2531 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 14:56:43.416969 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.416596 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j22g\" (UniqueName: \"kubernetes.io/projected/40eda5e0-b362-4b86-9ebe-87f141f0fee9-kube-api-access-9j22g\") pod \"network-check-source-7d88cb55d8-nt4bv\" (UID: \"40eda5e0-b362-4b86-9ebe-87f141f0fee9\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nt4bv" Jul 24 14:56:43.416969 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.416649 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls podName:31f123ed-b090-4d80-a46b-888032edc051 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:43.916616693 +0000 UTC m=+36.147093359 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls") pod "dns-default-2jdtw" (UID: "31f123ed-b090-4d80-a46b-888032edc051") : secret "dns-default-metrics-tls" not found Jul 24 14:56:43.416969 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.416676 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/595851c1-64b8-44ad-b819-f6b635101e2e-tmp\") pod \"insights-operator-5885fdb759-wjxvg\" (UID: \"595851c1-64b8-44ad-b819-f6b635101e2e\") " pod="openshift-insights/insights-operator-5885fdb759-wjxvg" Jul 24 14:56:43.416969 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.416751 2531 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 24 14:56:43.416969 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.416813 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert podName:020f32d2-c323-430c-b6a6-d97a25182801 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:43.916795211 +0000 UTC m=+36.147271873 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-dntl4" (UID: "020f32d2-c323-430c-b6a6-d97a25182801") : secret "networking-console-plugin-cert" not found Jul 24 14:56:43.416969 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.416833 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/595851c1-64b8-44ad-b819-f6b635101e2e-service-ca-bundle\") pod \"insights-operator-5885fdb759-wjxvg\" (UID: \"595851c1-64b8-44ad-b819-f6b635101e2e\") " pod="openshift-insights/insights-operator-5885fdb759-wjxvg" Jul 24 14:56:43.416969 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.416967 2531 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 14:56:43.417334 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.417043 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls podName:afe11285-00c7-47a8-a0d0-08acd3df5bda nodeName:}" failed. No retries permitted until 2026-07-24 14:56:43.917026256 +0000 UTC m=+36.147502931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls") pod "cluster-samples-operator-6748489656-vl525" (UID: "afe11285-00c7-47a8-a0d0-08acd3df5bda") : secret "samples-operator-tls" not found Jul 24 14:56:43.417334 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.417063 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31f123ed-b090-4d80-a46b-888032edc051-tmp-dir\") pod \"dns-default-2jdtw\" (UID: \"31f123ed-b090-4d80-a46b-888032edc051\") " pod="openshift-dns/dns-default-2jdtw" Jul 24 14:56:43.417334 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.417168 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/31f123ed-b090-4d80-a46b-888032edc051-config-volume\") pod \"dns-default-2jdtw\" (UID: \"31f123ed-b090-4d80-a46b-888032edc051\") " pod="openshift-dns/dns-default-2jdtw" Jul 24 14:56:43.417438 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.417391 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/91fda333-13a3-4954-8c20-e3c8c8bd4da3-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc\" (UID: \"91fda333-13a3-4954-8c20-e3c8c8bd4da3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc" Jul 24 14:56:43.418299 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.418277 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/91fda333-13a3-4954-8c20-e3c8c8bd4da3-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc\" (UID: \"91fda333-13a3-4954-8c20-e3c8c8bd4da3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc" Jul 24 14:56:43.418391 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.418282 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/595851c1-64b8-44ad-b819-f6b635101e2e-serving-cert\") pod \"insights-operator-5885fdb759-wjxvg\" (UID: \"595851c1-64b8-44ad-b819-f6b635101e2e\") " pod="openshift-insights/insights-operator-5885fdb759-wjxvg" Jul 24 14:56:43.419407 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.419390 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e5d1043-9361-463b-a342-bcb81a9f181d-cert\") pod \"ingress-canary-s4b7h\" (UID: \"9e5d1043-9361-463b-a342-bcb81a9f181d\") " pod="openshift-ingress-canary/ingress-canary-s4b7h" Jul 24 14:56:43.426161 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.426137 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlb4r\" (UniqueName: \"kubernetes.io/projected/31f123ed-b090-4d80-a46b-888032edc051-kube-api-access-qlb4r\") pod \"dns-default-2jdtw\" (UID: \"31f123ed-b090-4d80-a46b-888032edc051\") " pod="openshift-dns/dns-default-2jdtw" Jul 24 14:56:43.427393 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.427376 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsc6q\" (UniqueName: \"kubernetes.io/projected/91fda333-13a3-4954-8c20-e3c8c8bd4da3-kube-api-access-gsc6q\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc\" (UID: \"91fda333-13a3-4954-8c20-e3c8c8bd4da3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc" Jul 24 14:56:43.429156 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.429133 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54j49\" (UniqueName: \"kubernetes.io/projected/595851c1-64b8-44ad-b819-f6b635101e2e-kube-api-access-54j49\") pod \"insights-operator-5885fdb759-wjxvg\" (UID: \"595851c1-64b8-44ad-b819-f6b635101e2e\") " pod="openshift-insights/insights-operator-5885fdb759-wjxvg" Jul 24 14:56:43.429476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.429455 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j22g\" (UniqueName: \"kubernetes.io/projected/40eda5e0-b362-4b86-9ebe-87f141f0fee9-kube-api-access-9j22g\") pod \"network-check-source-7d88cb55d8-nt4bv\" (UID: \"40eda5e0-b362-4b86-9ebe-87f141f0fee9\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nt4bv" Jul 24 14:56:43.433038 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.432987 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlmtd\" (UniqueName: \"kubernetes.io/projected/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-kube-api-access-hlmtd\") pod \"cluster-monitoring-operator-548b489cb8-86bxn\" (UID: \"2872026a-6cfb-4c62-8ae5-b33297a9b1e7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn" Jul 24 14:56:43.434902 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.434879 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mw7p\" (UniqueName: \"kubernetes.io/projected/9e5d1043-9361-463b-a342-bcb81a9f181d-kube-api-access-9mw7p\") pod \"ingress-canary-s4b7h\" (UID: \"9e5d1043-9361-463b-a342-bcb81a9f181d\") " pod="openshift-ingress-canary/ingress-canary-s4b7h" Jul 24 14:56:43.441777 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.441758 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlsgj\" (UniqueName: \"kubernetes.io/projected/afe11285-00c7-47a8-a0d0-08acd3df5bda-kube-api-access-zlsgj\") pod \"cluster-samples-operator-6748489656-vl525\" (UID: \"afe11285-00c7-47a8-a0d0-08acd3df5bda\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525" Jul 24 14:56:43.450144 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.450122 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qkp5x" Jul 24 14:56:43.459359 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.459341 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" Jul 24 14:56:43.480401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.480374 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ndjm9" Jul 24 14:56:43.509984 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.509847 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:43.511714 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.511539 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-wjxvg" Jul 24 14:56:43.525046 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.524818 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nt4bv" Jul 24 14:56:43.541014 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.538521 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc" Jul 24 14:56:43.548611 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.548414 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s4b7h" Jul 24 14:56:43.603571 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.599843 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:43.668381 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.668277 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qkp5x"] Jul 24 14:56:43.679240 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.679161 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-77k6k"] Jul 24 14:56:43.713744 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.713673 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ndjm9"] Jul 24 14:56:43.724326 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:43.724298 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72091411_787f_478a_a97f_8ebc53464929.slice/crio-aa2dfea5e09f95fbd4ab336af7c808814f143b36364d926324be9de119b5b2e9 WatchSource:0}: Error finding container aa2dfea5e09f95fbd4ab336af7c808814f143b36364d926324be9de119b5b2e9: Status 404 returned error can't find the container with id aa2dfea5e09f95fbd4ab336af7c808814f143b36364d926324be9de119b5b2e9 Jul 24 14:56:43.774612 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.774568 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-wjxvg"] Jul 24 14:56:43.777562 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:43.777532 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod595851c1_64b8_44ad_b819_f6b635101e2e.slice/crio-bba30e490635518db75879db27f00d5659f5e46b662a430511aa818ac3cf0333 WatchSource:0}: Error finding container bba30e490635518db75879db27f00d5659f5e46b662a430511aa818ac3cf0333: Status 404 returned error can't find the container with id bba30e490635518db75879db27f00d5659f5e46b662a430511aa818ac3cf0333 Jul 24 14:56:43.783896 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.783821 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-nt4bv"] Jul 24 14:56:43.787026 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:43.787001 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40eda5e0_b362_4b86_9ebe_87f141f0fee9.slice/crio-f95415d262d74b350f54a80b1480da08e65ceef48b7c1c6685aff07098e2bdc1 WatchSource:0}: Error finding container f95415d262d74b350f54a80b1480da08e65ceef48b7c1c6685aff07098e2bdc1: Status 404 returned error can't find the container with id f95415d262d74b350f54a80b1480da08e65ceef48b7c1c6685aff07098e2bdc1 Jul 24 14:56:43.788429 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.788412 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s4b7h"] Jul 24 14:56:43.789295 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:43.789253 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e5d1043_9361_463b_a342_bcb81a9f181d.slice/crio-b8f658746848d401a958d3251f77587bdaa96c71f7e485a67fcca918d710a7af WatchSource:0}: Error finding container b8f658746848d401a958d3251f77587bdaa96c71f7e485a67fcca918d710a7af: Status 404 returned error can't find the container with id b8f658746848d401a958d3251f77587bdaa96c71f7e485a67fcca918d710a7af Jul 24 14:56:43.803587 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.803555 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc"] Jul 24 14:56:43.808299 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:56:43.808269 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91fda333_13a3_4954_8c20_e3c8c8bd4da3.slice/crio-21d584f5cfb314b38ff03b520e777b7756b562f957133665a92dd6cdead46dcf WatchSource:0}: Error finding container 21d584f5cfb314b38ff03b520e777b7756b562f957133665a92dd6cdead46dcf: Status 404 returned error can't find the container with id 21d584f5cfb314b38ff03b520e777b7756b562f957133665a92dd6cdead46dcf Jul 24 14:56:43.824526 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.824502 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:43.824624 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.824563 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:43.824760 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.824649 2531 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 14:56:43.824760 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.824668 2531 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-779758b8b8-dc8j5: secret "image-registry-tls" not found Jul 24 14:56:43.824760 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.824671 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:43.824760 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.824720 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls podName:888b99fe-7dd7-499b-9326-37b892a7ef56 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:44.824699833 +0000 UTC m=+37.055176499 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls") pod "image-registry-779758b8b8-dc8j5" (UID: "888b99fe-7dd7-499b-9326-37b892a7ef56") : secret "image-registry-tls" not found Jul 24 14:56:43.824760 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.824755 2531 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 14:56:43.824994 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.824769 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle podName:3ede17c9-5194-41b7-a4e6-55a258fe1a4e nodeName:}" failed. No retries permitted until 2026-07-24 14:56:44.824755077 +0000 UTC m=+37.055231739 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle") pod "router-default-776fc686df-t8g55" (UID: "3ede17c9-5194-41b7-a4e6-55a258fe1a4e") : configmap references non-existent config key: service-ca.crt Jul 24 14:56:43.824994 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.824831 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs podName:3ede17c9-5194-41b7-a4e6-55a258fe1a4e nodeName:}" failed. No retries permitted until 2026-07-24 14:56:44.824814737 +0000 UTC m=+37.055291415 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs") pod "router-default-776fc686df-t8g55" (UID: "3ede17c9-5194-41b7-a4e6-55a258fe1a4e") : secret "router-metrics-certs-default" not found Jul 24 14:56:43.925882 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.925834 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dntl4\" (UID: \"020f32d2-c323-430c-b6a6-d97a25182801\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dntl4" Jul 24 14:56:43.925882 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.925884 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vl525\" (UID: \"afe11285-00c7-47a8-a0d0-08acd3df5bda\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525" Jul 24 14:56:43.926117 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.925926 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls\") pod \"dns-default-2jdtw\" (UID: \"31f123ed-b090-4d80-a46b-888032edc051\") " pod="openshift-dns/dns-default-2jdtw" Jul 24 14:56:43.926117 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.925978 2531 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 24 14:56:43.926117 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.926039 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert podName:020f32d2-c323-430c-b6a6-d97a25182801 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:44.926023279 +0000 UTC m=+37.156499941 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-dntl4" (UID: "020f32d2-c323-430c-b6a6-d97a25182801") : secret "networking-console-plugin-cert" not found Jul 24 14:56:43.926117 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.926046 2531 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 14:56:43.926117 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.926058 2531 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 14:56:43.926117 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:43.925987 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-86bxn\" (UID: \"2872026a-6cfb-4c62-8ae5-b33297a9b1e7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn" Jul 24 14:56:43.926117 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.926062 2531 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 14:56:43.926117 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.926088 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls podName:2872026a-6cfb-4c62-8ae5-b33297a9b1e7 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:44.926073214 +0000 UTC m=+37.156549876 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-86bxn" (UID: "2872026a-6cfb-4c62-8ae5-b33297a9b1e7") : secret "cluster-monitoring-operator-tls" not found Jul 24 14:56:43.926117 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.926106 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls podName:afe11285-00c7-47a8-a0d0-08acd3df5bda nodeName:}" failed. No retries permitted until 2026-07-24 14:56:44.926098799 +0000 UTC m=+37.156575460 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls") pod "cluster-samples-operator-6748489656-vl525" (UID: "afe11285-00c7-47a8-a0d0-08acd3df5bda") : secret "samples-operator-tls" not found Jul 24 14:56:43.926422 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:43.926138 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls podName:31f123ed-b090-4d80-a46b-888032edc051 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:44.926132599 +0000 UTC m=+37.156609260 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls") pod "dns-default-2jdtw" (UID: "31f123ed-b090-4d80-a46b-888032edc051") : secret "dns-default-metrics-tls" not found Jul 24 14:56:44.276248 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:44.276218 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:56:44.279649 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:44.279625 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 14:56:44.492772 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:44.492729 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s4b7h" event={"ID":"9e5d1043-9361-463b-a342-bcb81a9f181d","Type":"ContainerStarted","Data":"b8f658746848d401a958d3251f77587bdaa96c71f7e485a67fcca918d710a7af"} Jul 24 14:56:44.494487 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:44.494445 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-wjxvg" event={"ID":"595851c1-64b8-44ad-b819-f6b635101e2e","Type":"ContainerStarted","Data":"bba30e490635518db75879db27f00d5659f5e46b662a430511aa818ac3cf0333"} Jul 24 14:56:44.499350 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:44.499271 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ndjm9" event={"ID":"72091411-787f-478a-a97f-8ebc53464929","Type":"ContainerStarted","Data":"aa2dfea5e09f95fbd4ab336af7c808814f143b36364d926324be9de119b5b2e9"} Jul 24 14:56:44.501286 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:44.501206 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nt4bv" event={"ID":"40eda5e0-b362-4b86-9ebe-87f141f0fee9","Type":"ContainerStarted","Data":"f95415d262d74b350f54a80b1480da08e65ceef48b7c1c6685aff07098e2bdc1"} Jul 24 14:56:44.504388 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:44.504313 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qkp5x" event={"ID":"079183d0-ba31-4d48-aa8f-e11069836062","Type":"ContainerStarted","Data":"759aeef940f6192ba54c540d4e2c75d082b4460ed1b544be9c01d1b287d27134"} Jul 24 14:56:44.506303 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:44.506230 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" event={"ID":"520066e2-8b42-4872-b257-20df38f77e37","Type":"ContainerStarted","Data":"7c8a500354ea256b31b886f19f04d34a5765e2607970ff5166e8af603cef7b76"} Jul 24 14:56:44.508420 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:44.508395 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc" event={"ID":"91fda333-13a3-4954-8c20-e3c8c8bd4da3","Type":"ContainerStarted","Data":"21d584f5cfb314b38ff03b520e777b7756b562f957133665a92dd6cdead46dcf"} Jul 24 14:56:44.837036 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:44.836997 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:44.837209 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:44.837100 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:44.837209 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:44.837143 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:44.837342 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:44.837286 2531 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 14:56:44.837393 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:44.837352 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs podName:3ede17c9-5194-41b7-a4e6-55a258fe1a4e nodeName:}" failed. No retries permitted until 2026-07-24 14:56:46.837332545 +0000 UTC m=+39.067809210 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs") pod "router-default-776fc686df-t8g55" (UID: "3ede17c9-5194-41b7-a4e6-55a258fe1a4e") : secret "router-metrics-certs-default" not found Jul 24 14:56:44.837814 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:44.837794 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle podName:3ede17c9-5194-41b7-a4e6-55a258fe1a4e nodeName:}" failed. No retries permitted until 2026-07-24 14:56:46.837776192 +0000 UTC m=+39.068252856 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle") pod "router-default-776fc686df-t8g55" (UID: "3ede17c9-5194-41b7-a4e6-55a258fe1a4e") : configmap references non-existent config key: service-ca.crt Jul 24 14:56:44.837930 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:44.837897 2531 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 14:56:44.837930 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:44.837909 2531 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-779758b8b8-dc8j5: secret "image-registry-tls" not found Jul 24 14:56:44.838045 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:44.837942 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls podName:888b99fe-7dd7-499b-9326-37b892a7ef56 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:46.837931024 +0000 UTC m=+39.068407689 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls") pod "image-registry-779758b8b8-dc8j5" (UID: "888b99fe-7dd7-499b-9326-37b892a7ef56") : secret "image-registry-tls" not found Jul 24 14:56:44.860936 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:44.860846 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:44.935933 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:44.935892 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:44.938600 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:44.938575 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls\") pod \"dns-default-2jdtw\" (UID: \"31f123ed-b090-4d80-a46b-888032edc051\") " pod="openshift-dns/dns-default-2jdtw" Jul 24 14:56:44.938743 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:44.938689 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-86bxn\" (UID: \"2872026a-6cfb-4c62-8ae5-b33297a9b1e7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn" Jul 24 14:56:44.938743 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:44.938707 2531 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 14:56:44.938850 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:44.938773 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls podName:31f123ed-b090-4d80-a46b-888032edc051 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:46.938753756 +0000 UTC m=+39.169230421 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls") pod "dns-default-2jdtw" (UID: "31f123ed-b090-4d80-a46b-888032edc051") : secret "dns-default-metrics-tls" not found Jul 24 14:56:44.938850 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:44.938843 2531 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 14:56:44.938984 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:44.938859 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dntl4\" (UID: \"020f32d2-c323-430c-b6a6-d97a25182801\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dntl4" Jul 24 14:56:44.938984 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:44.938910 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vl525\" (UID: \"afe11285-00c7-47a8-a0d0-08acd3df5bda\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525" Jul 24 14:56:44.938984 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:44.938924 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls podName:2872026a-6cfb-4c62-8ae5-b33297a9b1e7 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:46.938907429 +0000 UTC m=+39.169384109 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-86bxn" (UID: "2872026a-6cfb-4c62-8ae5-b33297a9b1e7") : secret "cluster-monitoring-operator-tls" not found Jul 24 14:56:44.938984 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:44.938983 2531 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 14:56:44.939179 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:44.939022 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls podName:afe11285-00c7-47a8-a0d0-08acd3df5bda nodeName:}" failed. No retries permitted until 2026-07-24 14:56:46.93900913 +0000 UTC m=+39.169485796 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls") pod "cluster-samples-operator-6748489656-vl525" (UID: "afe11285-00c7-47a8-a0d0-08acd3df5bda") : secret "samples-operator-tls" not found Jul 24 14:56:44.939179 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:44.939078 2531 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 24 14:56:44.939179 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:44.939111 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert podName:020f32d2-c323-430c-b6a6-d97a25182801 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:46.939101159 +0000 UTC m=+39.169577824 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-dntl4" (UID: "020f32d2-c323-430c-b6a6-d97a25182801") : secret "networking-console-plugin-cert" not found Jul 24 14:56:45.040323 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:45.040289 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:45.885015 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:45.884112 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:45.984014 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:45.983907 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:46.819989 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:46.819790 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:46.858066 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:46.858024 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:46.858245 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:46.858129 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:46.858245 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:46.858176 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:46.858245 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:46.858203 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle podName:3ede17c9-5194-41b7-a4e6-55a258fe1a4e nodeName:}" failed. No retries permitted until 2026-07-24 14:56:50.858182701 +0000 UTC m=+43.088659384 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle") pod "router-default-776fc686df-t8g55" (UID: "3ede17c9-5194-41b7-a4e6-55a258fe1a4e") : configmap references non-existent config key: service-ca.crt Jul 24 14:56:46.858405 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:46.858273 2531 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 14:56:46.858405 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:46.858294 2531 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 14:56:46.858405 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:46.858312 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs podName:3ede17c9-5194-41b7-a4e6-55a258fe1a4e nodeName:}" failed. No retries permitted until 2026-07-24 14:56:50.858300297 +0000 UTC m=+43.088776961 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs") pod "router-default-776fc686df-t8g55" (UID: "3ede17c9-5194-41b7-a4e6-55a258fe1a4e") : secret "router-metrics-certs-default" not found Jul 24 14:56:46.858405 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:46.858315 2531 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-779758b8b8-dc8j5: secret "image-registry-tls" not found Jul 24 14:56:46.858405 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:46.858367 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls podName:888b99fe-7dd7-499b-9326-37b892a7ef56 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:50.858350023 +0000 UTC m=+43.088826707 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls") pod "image-registry-779758b8b8-dc8j5" (UID: "888b99fe-7dd7-499b-9326-37b892a7ef56") : secret "image-registry-tls" not found Jul 24 14:56:46.921360 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:46.921324 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:46.959320 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:46.959289 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dntl4\" (UID: \"020f32d2-c323-430c-b6a6-d97a25182801\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dntl4" Jul 24 14:56:46.959480 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:46.959327 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vl525\" (UID: \"afe11285-00c7-47a8-a0d0-08acd3df5bda\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525" Jul 24 14:56:46.959480 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:46.959384 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls\") pod \"dns-default-2jdtw\" (UID: \"31f123ed-b090-4d80-a46b-888032edc051\") " pod="openshift-dns/dns-default-2jdtw" Jul 24 14:56:46.959480 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:46.959447 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-86bxn\" (UID: \"2872026a-6cfb-4c62-8ae5-b33297a9b1e7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn" Jul 24 14:56:46.959480 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:46.959473 2531 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 24 14:56:46.959644 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:46.959526 2531 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 14:56:46.959644 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:46.959541 2531 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 14:56:46.959644 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:46.959545 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert podName:020f32d2-c323-430c-b6a6-d97a25182801 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:50.959524233 +0000 UTC m=+43.190000918 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-dntl4" (UID: "020f32d2-c323-430c-b6a6-d97a25182801") : secret "networking-console-plugin-cert" not found Jul 24 14:56:46.959644 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:46.959540 2531 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 14:56:46.959644 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:46.959579 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls podName:2872026a-6cfb-4c62-8ae5-b33297a9b1e7 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:50.959564953 +0000 UTC m=+43.190041614 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-86bxn" (UID: "2872026a-6cfb-4c62-8ae5-b33297a9b1e7") : secret "cluster-monitoring-operator-tls" not found Jul 24 14:56:46.959644 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:46.959594 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls podName:afe11285-00c7-47a8-a0d0-08acd3df5bda nodeName:}" failed. No retries permitted until 2026-07-24 14:56:50.959586594 +0000 UTC m=+43.190063256 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls") pod "cluster-samples-operator-6748489656-vl525" (UID: "afe11285-00c7-47a8-a0d0-08acd3df5bda") : secret "samples-operator-tls" not found Jul 24 14:56:46.959644 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:46.959608 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls podName:31f123ed-b090-4d80-a46b-888032edc051 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:50.959600907 +0000 UTC m=+43.190077570 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls") pod "dns-default-2jdtw" (UID: "31f123ed-b090-4d80-a46b-888032edc051") : secret "dns-default-metrics-tls" not found Jul 24 14:56:48.528699 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:48.528652 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:48.630943 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:48.630903 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:50.411231 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:50.411135 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:50.511231 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:50.511192 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:50.895576 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:50.895536 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:50.895774 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:50.895597 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:50.895774 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:50.895668 2531 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 14:56:50.895774 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:50.895688 2531 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-779758b8b8-dc8j5: secret "image-registry-tls" not found Jul 24 14:56:50.895774 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:50.895700 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:50.895774 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:50.895753 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls podName:888b99fe-7dd7-499b-9326-37b892a7ef56 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:58.8957303 +0000 UTC m=+51.126206971 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls") pod "image-registry-779758b8b8-dc8j5" (UID: "888b99fe-7dd7-499b-9326-37b892a7ef56") : secret "image-registry-tls" not found Jul 24 14:56:50.895774 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:50.895756 2531 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 14:56:50.896047 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:50.895819 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle podName:3ede17c9-5194-41b7-a4e6-55a258fe1a4e nodeName:}" failed. No retries permitted until 2026-07-24 14:56:58.895802652 +0000 UTC m=+51.126279315 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle") pod "router-default-776fc686df-t8g55" (UID: "3ede17c9-5194-41b7-a4e6-55a258fe1a4e") : configmap references non-existent config key: service-ca.crt Jul 24 14:56:50.896047 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:50.895840 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs podName:3ede17c9-5194-41b7-a4e6-55a258fe1a4e nodeName:}" failed. No retries permitted until 2026-07-24 14:56:58.895832814 +0000 UTC m=+51.126309477 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs") pod "router-default-776fc686df-t8g55" (UID: "3ede17c9-5194-41b7-a4e6-55a258fe1a4e") : secret "router-metrics-certs-default" not found Jul 24 14:56:50.996483 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:50.996447 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-86bxn\" (UID: \"2872026a-6cfb-4c62-8ae5-b33297a9b1e7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn" Jul 24 14:56:50.996663 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:50.996531 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dntl4\" (UID: \"020f32d2-c323-430c-b6a6-d97a25182801\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dntl4" Jul 24 14:56:50.996663 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:50.996554 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vl525\" (UID: \"afe11285-00c7-47a8-a0d0-08acd3df5bda\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525" Jul 24 14:56:50.996663 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:50.996588 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls\") pod \"dns-default-2jdtw\" (UID: \"31f123ed-b090-4d80-a46b-888032edc051\") " pod="openshift-dns/dns-default-2jdtw" Jul 24 14:56:50.996663 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:50.996610 2531 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 14:56:50.996833 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:50.996681 2531 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 14:56:50.996833 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:50.996687 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls podName:2872026a-6cfb-4c62-8ae5-b33297a9b1e7 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:58.996669026 +0000 UTC m=+51.227145688 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-86bxn" (UID: "2872026a-6cfb-4c62-8ae5-b33297a9b1e7") : secret "cluster-monitoring-operator-tls" not found Jul 24 14:56:50.996833 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:50.996682 2531 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 24 14:56:50.996833 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:50.996743 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls podName:afe11285-00c7-47a8-a0d0-08acd3df5bda nodeName:}" failed. No retries permitted until 2026-07-24 14:56:58.996730373 +0000 UTC m=+51.227207035 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls") pod "cluster-samples-operator-6748489656-vl525" (UID: "afe11285-00c7-47a8-a0d0-08acd3df5bda") : secret "samples-operator-tls" not found Jul 24 14:56:50.996833 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:50.996682 2531 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 14:56:50.996833 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:50.996779 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert podName:020f32d2-c323-430c-b6a6-d97a25182801 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:58.996764417 +0000 UTC m=+51.227241084 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-dntl4" (UID: "020f32d2-c323-430c-b6a6-d97a25182801") : secret "networking-console-plugin-cert" not found Jul 24 14:56:50.996833 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:50.996801 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls podName:31f123ed-b090-4d80-a46b-888032edc051 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:58.996791202 +0000 UTC m=+51.227267863 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls") pod "dns-default-2jdtw" (UID: "31f123ed-b090-4d80-a46b-888032edc051") : secret "dns-default-metrics-tls" not found Jul 24 14:56:53.534997 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:53.534958 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc" event={"ID":"91fda333-13a3-4954-8c20-e3c8c8bd4da3","Type":"ContainerStarted","Data":"ff5e22fa0eae843794b50eac9d79df076552ecd7619c62b1ee322bc7843bfae4"} Jul 24 14:56:53.538273 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:53.538244 2531 generic.go:356] "Generic (PLEG): container finished" podID="02657d23-d776-4da0-939a-e1f8893c5904" containerID="5d106d6008e5f744eba16adcd55e94745cfc85c387f89c9d68e8284dc880a3e5" exitCode=0 Jul 24 14:56:53.538409 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:53.538313 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vgqjg" event={"ID":"02657d23-d776-4da0-939a-e1f8893c5904","Type":"ContainerDied","Data":"5d106d6008e5f744eba16adcd55e94745cfc85c387f89c9d68e8284dc880a3e5"} Jul 24 14:56:53.540709 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:53.540686 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s4b7h" event={"ID":"9e5d1043-9361-463b-a342-bcb81a9f181d","Type":"ContainerStarted","Data":"11aaf63d8f48d41ea296c197577eab1095183939263e0978b63f57cd7e22b747"} Jul 24 14:56:53.542693 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:53.542349 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-wjxvg" event={"ID":"595851c1-64b8-44ad-b819-f6b635101e2e","Type":"ContainerStarted","Data":"716f76777e30be7eb338b356f7dbb86000c46e6a6491f1e1a94af600b3cf1793"} Jul 24 14:56:53.546089 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:53.546067 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ndjm9" event={"ID":"72091411-787f-478a-a97f-8ebc53464929","Type":"ContainerStarted","Data":"1a3e6aa020222b37d5fbd090d996eb2348e658fda2b2dee91bde37d431540f8c"} Jul 24 14:56:53.547624 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:53.547593 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nt4bv" event={"ID":"40eda5e0-b362-4b86-9ebe-87f141f0fee9","Type":"ContainerStarted","Data":"5d869a5871377369ea1ae706463b8d00a56f37b04e75b2fbb7e54784ad0eeabc"} Jul 24 14:56:53.549132 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:53.549109 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qkp5x" event={"ID":"079183d0-ba31-4d48-aa8f-e11069836062","Type":"ContainerStarted","Data":"6782ae44238d268fd438f8812fd29dee44052f09e0348a0a50010a5960cfa82f"} Jul 24 14:56:53.550828 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:53.550810 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/0.log" Jul 24 14:56:53.550944 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:53.550849 2531 generic.go:356] "Generic (PLEG): container finished" podID="520066e2-8b42-4872-b257-20df38f77e37" containerID="48e144e29d5df2160ec3bcfd2f723b47c45277e7fc6a3fd258ae8426e58efb6c" exitCode=255 Jul 24 14:56:53.550944 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:53.550908 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" event={"ID":"520066e2-8b42-4872-b257-20df38f77e37","Type":"ContainerDied","Data":"48e144e29d5df2160ec3bcfd2f723b47c45277e7fc6a3fd258ae8426e58efb6c"} Jul 24 14:56:53.551133 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:53.551118 2531 scope.go:117] "RemoveContainer" containerID="48e144e29d5df2160ec3bcfd2f723b47c45277e7fc6a3fd258ae8426e58efb6c" Jul 24 14:56:53.574352 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:53.574308 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc" podStartSLOduration=21.676161612 podStartE2EDuration="30.574290709s" totalImagesPullingTime="8.898129097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:56:53.568523142 +0000 UTC m=+45.798999830" watchObservedRunningTime="2026-07-24 14:56:53.574290709 +0000 UTC m=+45.804767394" Jul 24 14:56:53.591971 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:53.591386 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:53.600857 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:53.600482 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qkp5x" podStartSLOduration=21.647602565 podStartE2EDuration="30.600465455s" totalImagesPullingTime="8.95286289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:56:53.599438327 +0000 UTC m=+45.829915016" watchObservedRunningTime="2026-07-24 14:56:53.600465455 +0000 UTC m=+45.830942140" Jul 24 14:56:53.625081 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:53.625034 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-ndjm9" podStartSLOduration=21.652034147 podStartE2EDuration="30.625018575s" totalImagesPullingTime="8.972984428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:56:53.624172479 +0000 UTC m=+45.854649167" watchObservedRunningTime="2026-07-24 14:56:53.625018575 +0000 UTC m=+45.855495260" Jul 24 14:56:53.668053 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:53.667999 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-wjxvg" podStartSLOduration=21.770904747 podStartE2EDuration="30.667978846s" totalImagesPullingTime="8.897074099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:56:53.666685528 +0000 UTC m=+45.897162230" watchObservedRunningTime="2026-07-24 14:56:53.667978846 +0000 UTC m=+45.898455531" Jul 24 14:56:53.687530 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:53.686258 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:53.738165 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:53.737814 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nt4bv" podStartSLOduration=21.873869704 podStartE2EDuration="30.737794697s" totalImagesPullingTime="8.863924993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:56:53.722221356 +0000 UTC m=+45.952698041" watchObservedRunningTime="2026-07-24 14:56:53.737794697 +0000 UTC m=+45.968271382" Jul 24 14:56:53.785586 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:53.785487 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:54.142353 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.142224 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:54.200066 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.200031 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:54.502888 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.502843 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:54.555928 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.555899 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 14:56:54.556323 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.556296 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/0.log" Jul 24 14:56:54.556375 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.556326 2531 generic.go:356] "Generic (PLEG): container finished" podID="520066e2-8b42-4872-b257-20df38f77e37" containerID="48db6626fe5e6625f72db8c795a865191a0c2cf4f31a98a3501ce023e1c48630" exitCode=255 Jul 24 14:56:54.556420 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.556385 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" event={"ID":"520066e2-8b42-4872-b257-20df38f77e37","Type":"ContainerDied","Data":"48db6626fe5e6625f72db8c795a865191a0c2cf4f31a98a3501ce023e1c48630"} Jul 24 14:56:54.556478 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.556448 2531 scope.go:117] "RemoveContainer" containerID="48e144e29d5df2160ec3bcfd2f723b47c45277e7fc6a3fd258ae8426e58efb6c" Jul 24 14:56:54.556656 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.556637 2531 scope.go:117] "RemoveContainer" containerID="48db6626fe5e6625f72db8c795a865191a0c2cf4f31a98a3501ce023e1c48630" Jul 24 14:56:54.556847 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:54.556821 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-77k6k_openshift-console-operator(520066e2-8b42-4872-b257-20df38f77e37)\"" pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" podUID="520066e2-8b42-4872-b257-20df38f77e37" Jul 24 14:56:54.559750 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.559728 2531 generic.go:356] "Generic (PLEG): container finished" podID="02657d23-d776-4da0-939a-e1f8893c5904" containerID="47d8e9ae1129c5b35f679ad5027e0593d54c5d5264235a1a0ba48f4b48ba5c2c" exitCode=0 Jul 24 14:56:54.559838 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.559810 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vgqjg" event={"ID":"02657d23-d776-4da0-939a-e1f8893c5904","Type":"ContainerDied","Data":"47d8e9ae1129c5b35f679ad5027e0593d54c5d5264235a1a0ba48f4b48ba5c2c"} Jul 24 14:56:54.574979 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.574936 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-s4b7h" podStartSLOduration=2.662427591 podStartE2EDuration="11.574922655s" totalImagesPullingTime="8.912495064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:56:53.738144478 +0000 UTC m=+45.968621163" watchObservedRunningTime="2026-07-24 14:56:54.574922655 +0000 UTC m=+46.805399338" Jul 24 14:56:54.578698 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.578679 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:54.651333 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.651307 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:54.857338 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.857257 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-p9x5t"] Jul 24 14:56:54.879369 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.879304 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p9x5t"] Jul 24 14:56:54.879535 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.879425 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:56:54.879813 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.879783 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:54.882355 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.882337 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 24 14:56:54.882355 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.882350 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-psqz4\"" Jul 24 14:56:54.882538 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.882377 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 24 14:56:54.934214 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.934186 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5d5g\" (UniqueName: \"kubernetes.io/projected/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-kube-api-access-w5d5g\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:56:54.934338 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.934309 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-crio-socket\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:56:54.934390 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.934355 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:56:54.934489 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.934472 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-data-volume\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:56:54.934528 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.934497 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:56:54.953167 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:54.953135 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:55.035802 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:55.035768 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-data-volume\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:56:55.035802 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:55.035799 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:56:55.036058 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:55.035848 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5d5g\" (UniqueName: \"kubernetes.io/projected/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-kube-api-access-w5d5g\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:56:55.036058 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:55.035923 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-crio-socket\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:56:55.036058 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:55.035922 2531 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 14:56:55.036058 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:55.035952 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:56:55.036058 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:55.035987 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-kube-rbac-proxy-cm podName:54da61a7-9d56-41ec-8536-b1e4bfe30fb3 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:55.535970401 +0000 UTC m=+47.766447065 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p9x5t" (UID: "54da61a7-9d56-41ec-8536-b1e4bfe30fb3") : configmap "kube-rbac-proxy" not found Jul 24 14:56:55.036058 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:55.036038 2531 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 14:56:55.036315 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:55.036083 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-insights-runtime-extractor-tls podName:54da61a7-9d56-41ec-8536-b1e4bfe30fb3 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:55.536068452 +0000 UTC m=+47.766545165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9x5t" (UID: "54da61a7-9d56-41ec-8536-b1e4bfe30fb3") : secret "insights-runtime-extractor-tls" not found Jul 24 14:56:55.036315 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:55.036106 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-crio-socket\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:56:55.036315 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:55.036151 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-data-volume\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:56:55.038981 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:55.038957 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:55.046792 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:55.046773 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5d5g\" (UniqueName: \"kubernetes.io/projected/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-kube-api-access-w5d5g\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:56:55.139062 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:55.138976 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:55.544439 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:55.544402 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:56:55.544600 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:55.544481 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:56:55.544600 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:55.544552 2531 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 14:56:55.544600 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:55.544573 2531 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 14:56:55.544701 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:55.544615 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-kube-rbac-proxy-cm podName:54da61a7-9d56-41ec-8536-b1e4bfe30fb3 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:56.544601109 +0000 UTC m=+48.775077772 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p9x5t" (UID: "54da61a7-9d56-41ec-8536-b1e4bfe30fb3") : configmap "kube-rbac-proxy" not found Jul 24 14:56:55.544701 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:55.544647 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-insights-runtime-extractor-tls podName:54da61a7-9d56-41ec-8536-b1e4bfe30fb3 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:56.544630653 +0000 UTC m=+48.775107328 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9x5t" (UID: "54da61a7-9d56-41ec-8536-b1e4bfe30fb3") : secret "insights-runtime-extractor-tls" not found Jul 24 14:56:55.552413 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:55.552379 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:55.564935 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:55.564912 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 14:56:55.565333 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:55.565253 2531 scope.go:117] "RemoveContainer" containerID="48db6626fe5e6625f72db8c795a865191a0c2cf4f31a98a3501ce023e1c48630" Jul 24 14:56:55.565460 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:55.565439 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-77k6k_openshift-console-operator(520066e2-8b42-4872-b257-20df38f77e37)\"" pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" podUID="520066e2-8b42-4872-b257-20df38f77e37" Jul 24 14:56:55.568584 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:55.568559 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vgqjg" event={"ID":"02657d23-d776-4da0-939a-e1f8893c5904","Type":"ContainerStarted","Data":"363b0db9acc01e5a948e2dcf075f8a3bf5fdaf7576ad4f6feecf3d4910658146"} Jul 24 14:56:55.610218 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:55.610169 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vgqjg" podStartSLOduration=3.888564374 podStartE2EDuration="46.610150376s" totalImagesPullingTime="42.721586002s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:56:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-24 14:56:55.60920101 +0000 UTC m=+47.839677693" watchObservedRunningTime="2026-07-24 14:56:55.610150376 +0000 UTC m=+47.840627061" Jul 24 14:56:55.652044 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:55.652017 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:55.751989 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:55.751952 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:55.855333 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:55.855260 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:56.243748 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:56.243724 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hcxz7_bdaadab4-2a42-46d7-8ca6-24881a9d54d0/dns-node-resolver/0.log" Jul 24 14:56:56.536421 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:56.536330 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:56.554493 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:56.554465 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:56:56.554631 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:56.554545 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:56:56.554631 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:56.554601 2531 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 14:56:56.554702 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:56.554639 2531 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 14:56:56.554702 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:56.554658 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-insights-runtime-extractor-tls podName:54da61a7-9d56-41ec-8536-b1e4bfe30fb3 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:58.554643979 +0000 UTC m=+50.785120641 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9x5t" (UID: "54da61a7-9d56-41ec-8536-b1e4bfe30fb3") : secret "insights-runtime-extractor-tls" not found Jul 24 14:56:56.554702 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:56.554681 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-kube-rbac-proxy-cm podName:54da61a7-9d56-41ec-8536-b1e4bfe30fb3 nodeName:}" failed. No retries permitted until 2026-07-24 14:56:58.554668344 +0000 UTC m=+50.785145007 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p9x5t" (UID: "54da61a7-9d56-41ec-8536-b1e4bfe30fb3") : configmap "kube-rbac-proxy" not found Jul 24 14:56:56.606010 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:56.605971 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:57.095817 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:57.095786 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:57.197052 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:57.197018 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:57.445246 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:57.445218 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pbnwh_91255a26-e33b-408d-81d1-b8ec24d1e7d6/node-ca/0.log" Jul 24 14:56:58.065265 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:58.065235 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:58.255100 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:58.255075 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s4b7h_9e5d1043-9361-463b-a342-bcb81a9f181d/serve-healthcheck-canary/0.log" Jul 24 14:56:58.571064 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:58.571033 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:56:58.571223 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:58.571096 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:56:58.571223 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:58.571168 2531 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 14:56:58.571223 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:58.571187 2531 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 14:56:58.571319 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:58.571230 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-kube-rbac-proxy-cm podName:54da61a7-9d56-41ec-8536-b1e4bfe30fb3 nodeName:}" failed. No retries permitted until 2026-07-24 14:57:02.571214236 +0000 UTC m=+54.801690897 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p9x5t" (UID: "54da61a7-9d56-41ec-8536-b1e4bfe30fb3") : configmap "kube-rbac-proxy" not found Jul 24 14:56:58.571319 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:58.571244 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-insights-runtime-extractor-tls podName:54da61a7-9d56-41ec-8536-b1e4bfe30fb3 nodeName:}" failed. No retries permitted until 2026-07-24 14:57:02.571237869 +0000 UTC m=+54.801714530 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9x5t" (UID: "54da61a7-9d56-41ec-8536-b1e4bfe30fb3") : secret "insights-runtime-extractor-tls" not found Jul 24 14:56:58.841847 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:58.841754 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:56:58.974560 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:58.974528 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:56:58.974714 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:58.974588 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:58.974714 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:58.974686 2531 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 14:56:58.974714 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:58.974702 2531 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-779758b8b8-dc8j5: secret "image-registry-tls" not found Jul 24 14:56:58.974714 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:58.974703 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:56:58.974851 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:58.974742 2531 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 14:56:58.974851 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:58.974760 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls podName:888b99fe-7dd7-499b-9326-37b892a7ef56 nodeName:}" failed. No retries permitted until 2026-07-24 14:57:14.974741402 +0000 UTC m=+67.205218064 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls") pod "image-registry-779758b8b8-dc8j5" (UID: "888b99fe-7dd7-499b-9326-37b892a7ef56") : secret "image-registry-tls" not found Jul 24 14:56:58.974851 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:58.974809 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs podName:3ede17c9-5194-41b7-a4e6-55a258fe1a4e nodeName:}" failed. No retries permitted until 2026-07-24 14:57:14.974791117 +0000 UTC m=+67.205267780 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs") pod "router-default-776fc686df-t8g55" (UID: "3ede17c9-5194-41b7-a4e6-55a258fe1a4e") : secret "router-metrics-certs-default" not found Jul 24 14:56:58.974851 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:58.974825 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle podName:3ede17c9-5194-41b7-a4e6-55a258fe1a4e nodeName:}" failed. No retries permitted until 2026-07-24 14:57:14.97481843 +0000 UTC m=+67.205295092 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle") pod "router-default-776fc686df-t8g55" (UID: "3ede17c9-5194-41b7-a4e6-55a258fe1a4e") : configmap references non-existent config key: service-ca.crt Jul 24 14:56:59.076141 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:59.076097 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dntl4\" (UID: \"020f32d2-c323-430c-b6a6-d97a25182801\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dntl4" Jul 24 14:56:59.076141 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:59.076131 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vl525\" (UID: \"afe11285-00c7-47a8-a0d0-08acd3df5bda\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525" Jul 24 14:56:59.076618 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:59.076164 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls\") pod \"dns-default-2jdtw\" (UID: \"31f123ed-b090-4d80-a46b-888032edc051\") " pod="openshift-dns/dns-default-2jdtw" Jul 24 14:56:59.076618 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:56:59.076221 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-86bxn\" (UID: \"2872026a-6cfb-4c62-8ae5-b33297a9b1e7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn" Jul 24 14:56:59.076618 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:59.076234 2531 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 24 14:56:59.076618 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:59.076270 2531 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 14:56:59.076618 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:59.076286 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert podName:020f32d2-c323-430c-b6a6-d97a25182801 nodeName:}" failed. No retries permitted until 2026-07-24 14:57:15.076272298 +0000 UTC m=+67.306748960 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-dntl4" (UID: "020f32d2-c323-430c-b6a6-d97a25182801") : secret "networking-console-plugin-cert" not found Jul 24 14:56:59.076618 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:59.076326 2531 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 14:56:59.076618 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:59.076333 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls podName:afe11285-00c7-47a8-a0d0-08acd3df5bda nodeName:}" failed. No retries permitted until 2026-07-24 14:57:15.076318381 +0000 UTC m=+67.306795046 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls") pod "cluster-samples-operator-6748489656-vl525" (UID: "afe11285-00c7-47a8-a0d0-08acd3df5bda") : secret "samples-operator-tls" not found Jul 24 14:56:59.076618 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:59.076334 2531 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 14:56:59.076618 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:59.076368 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls podName:31f123ed-b090-4d80-a46b-888032edc051 nodeName:}" failed. No retries permitted until 2026-07-24 14:57:15.076354634 +0000 UTC m=+67.306831318 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls") pod "dns-default-2jdtw" (UID: "31f123ed-b090-4d80-a46b-888032edc051") : secret "dns-default-metrics-tls" not found Jul 24 14:56:59.076618 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:56:59.076384 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls podName:2872026a-6cfb-4c62-8ae5-b33297a9b1e7 nodeName:}" failed. No retries permitted until 2026-07-24 14:57:15.076373695 +0000 UTC m=+67.306850359 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-86bxn" (UID: "2872026a-6cfb-4c62-8ae5-b33297a9b1e7") : secret "cluster-monitoring-operator-tls" not found Jul 24 14:57:01.937887 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:01.937836 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:02.038562 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:02.038500 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:02.138948 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:02.138902 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:02.239428 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:02.239340 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:02.339643 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:02.339613 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:02.406056 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:02.406025 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:02.607973 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:02.607882 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:57:02.608141 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:02.608008 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:57:02.608141 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:57:02.608109 2531 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 14:57:02.608262 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:57:02.608157 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-insights-runtime-extractor-tls podName:54da61a7-9d56-41ec-8536-b1e4bfe30fb3 nodeName:}" failed. No retries permitted until 2026-07-24 14:57:10.608143817 +0000 UTC m=+62.838620483 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9x5t" (UID: "54da61a7-9d56-41ec-8536-b1e4bfe30fb3") : secret "insights-runtime-extractor-tls" not found Jul 24 14:57:02.608539 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:02.608517 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:57:02.728425 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:02.728391 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:03.460430 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:03.460392 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" Jul 24 14:57:03.460844 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:03.460445 2531 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" Jul 24 14:57:03.460844 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:03.460772 2531 scope.go:117] "RemoveContainer" containerID="48db6626fe5e6625f72db8c795a865191a0c2cf4f31a98a3501ce023e1c48630" Jul 24 14:57:03.460963 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:57:03.460933 2531 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-77k6k_openshift-console-operator(520066e2-8b42-4872-b257-20df38f77e37)\"" pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" podUID="520066e2-8b42-4872-b257-20df38f77e37" Jul 24 14:57:03.936708 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:03.936675 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:04.112677 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:04.112646 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:04.314803 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:04.314724 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:04.521463 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:04.521426 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:04.918549 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:04.918506 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:05.315709 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:05.315667 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:05.717940 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:05.717911 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:05.960759 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:05.960715 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:06.119620 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:06.119542 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:06.518577 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:06.518539 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:06.722803 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:06.722768 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:07.047788 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:07.047754 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:07.147519 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:07.147484 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:07.318513 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:07.318447 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:07.920269 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:07.920235 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:08.267987 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:08.267913 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:08.369138 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:08.369106 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:08.523114 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:08.523039 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:09.364697 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:09.364668 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:09.484131 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:09.484103 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-f74fd" Jul 24 14:57:09.521727 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:09.521689 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:09.723050 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:09.723017 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:09.862003 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:09.861982 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:10.037156 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:10.037081 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:10.136898 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:10.136858 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:10.312085 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:10.312007 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:10.515088 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:10.515051 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:10.675003 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:10.674978 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:57:10.677309 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:10.677291 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/54da61a7-9d56-41ec-8536-b1e4bfe30fb3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9x5t\" (UID: \"54da61a7-9d56-41ec-8536-b1e4bfe30fb3\") " pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:57:10.791247 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:10.791219 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-psqz4\"" Jul 24 14:57:10.799207 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:10.799189 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p9x5t" Jul 24 14:57:10.923170 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:10.923141 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p9x5t"] Jul 24 14:57:10.926277 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:57:10.926213 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54da61a7_9d56_41ec_8536_b1e4bfe30fb3.slice/crio-75020af975346b897671859da5cf0e2ac05af040760048e4fedfd6305f79e9b8 WatchSource:0}: Error finding container 75020af975346b897671859da5cf0e2ac05af040760048e4fedfd6305f79e9b8: Status 404 returned error can't find the container with id 75020af975346b897671859da5cf0e2ac05af040760048e4fedfd6305f79e9b8 Jul 24 14:57:11.120034 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:11.120003 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:11.524752 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:11.524711 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:11.621785 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:11.621748 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p9x5t" event={"ID":"54da61a7-9d56-41ec-8536-b1e4bfe30fb3","Type":"ContainerStarted","Data":"75020af975346b897671859da5cf0e2ac05af040760048e4fedfd6305f79e9b8"} Jul 24 14:57:11.918363 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:11.918263 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:12.301124 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:12.301089 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:12.517396 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:12.517346 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:12.715965 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:12.715926 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:12.896962 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:12.896925 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4rz7\" (UniqueName: \"kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7\") pod \"network-check-target-mjqx9\" (UID: \"ec175d02-f2a5-4492-8133-e642a367da32\") " pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:57:12.897139 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:12.896974 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs\") pod \"network-metrics-daemon-xgswn\" (UID: \"a9494b1d-f2bb-408b-8283-7d622ea5e450\") " pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:57:12.899364 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:12.899336 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 14:57:12.899954 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:12.899924 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4rz7\" (UniqueName: \"kubernetes.io/projected/ec175d02-f2a5-4492-8133-e642a367da32-kube-api-access-z4rz7\") pod \"network-check-target-mjqx9\" (UID: \"ec175d02-f2a5-4492-8133-e642a367da32\") " pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:57:12.909739 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:12.909711 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9494b1d-f2bb-408b-8283-7d622ea5e450-metrics-certs\") pod \"network-metrics-daemon-xgswn\" (UID: \"a9494b1d-f2bb-408b-8283-7d622ea5e450\") " pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:57:12.983219 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:12.983150 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5tbx2\"" Jul 24 14:57:12.987265 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:12.987238 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j76t9\"" Jul 24 14:57:12.991031 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:12.991009 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xgswn" Jul 24 14:57:12.996113 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:12.996088 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:57:12.997654 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:12.997631 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret\") pod \"global-pull-secret-syncer-6jf8n\" (UID: \"441a2a3d-5df6-4a88-92cc-63d18ba4310f\") " pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:57:13.000104 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:13.000082 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 14:57:13.013065 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:13.010775 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/441a2a3d-5df6-4a88-92cc-63d18ba4310f-original-pull-secret\") pod \"global-pull-secret-syncer-6jf8n\" (UID: \"441a2a3d-5df6-4a88-92cc-63d18ba4310f\") " pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:57:13.090267 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:13.090196 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jf8n" Jul 24 14:57:13.140482 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:13.140391 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xgswn"] Jul 24 14:57:13.155204 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:13.155182 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mjqx9"] Jul 24 14:57:15.015207 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.015170 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:57:15.015595 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.015234 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:57:15.015595 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.015301 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:57:15.015968 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.015946 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-service-ca-bundle\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:57:15.017753 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.017728 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ede17c9-5194-41b7-a4e6-55a258fe1a4e-metrics-certs\") pod \"router-default-776fc686df-t8g55\" (UID: \"3ede17c9-5194-41b7-a4e6-55a258fe1a4e\") " pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:57:15.017846 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.017762 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls\") pod \"image-registry-779758b8b8-dc8j5\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:57:15.077758 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:57:15.077723 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec175d02_f2a5_4492_8133_e642a367da32.slice/crio-e2994c4e1083b7ad06ce3d84e2bc5e78bdad42bcbc0b83a12b98fea2b7292ed6 WatchSource:0}: Error finding container e2994c4e1083b7ad06ce3d84e2bc5e78bdad42bcbc0b83a12b98fea2b7292ed6: Status 404 returned error can't find the container with id e2994c4e1083b7ad06ce3d84e2bc5e78bdad42bcbc0b83a12b98fea2b7292ed6 Jul 24 14:57:15.078934 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:57:15.078910 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9494b1d_f2bb_408b_8283_7d622ea5e450.slice/crio-8050d0d98a19d0e0c06e173cad9207d502145f25524d7edf428ec8a3ca1b6ca9 WatchSource:0}: Error finding container 8050d0d98a19d0e0c06e173cad9207d502145f25524d7edf428ec8a3ca1b6ca9: Status 404 returned error can't find the container with id 8050d0d98a19d0e0c06e173cad9207d502145f25524d7edf428ec8a3ca1b6ca9 Jul 24 14:57:15.116343 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.116296 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-86bxn\" (UID: \"2872026a-6cfb-4c62-8ae5-b33297a9b1e7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn" Jul 24 14:57:15.116519 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.116385 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dntl4\" (UID: \"020f32d2-c323-430c-b6a6-d97a25182801\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dntl4" Jul 24 14:57:15.116519 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.116417 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vl525\" (UID: \"afe11285-00c7-47a8-a0d0-08acd3df5bda\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525" Jul 24 14:57:15.116519 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.116464 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls\") pod \"dns-default-2jdtw\" (UID: \"31f123ed-b090-4d80-a46b-888032edc051\") " pod="openshift-dns/dns-default-2jdtw" Jul 24 14:57:15.119231 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.119199 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31f123ed-b090-4d80-a46b-888032edc051-metrics-tls\") pod \"dns-default-2jdtw\" (UID: \"31f123ed-b090-4d80-a46b-888032edc051\") " pod="openshift-dns/dns-default-2jdtw" Jul 24 14:57:15.119409 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.119391 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/afe11285-00c7-47a8-a0d0-08acd3df5bda-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vl525\" (UID: \"afe11285-00c7-47a8-a0d0-08acd3df5bda\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525" Jul 24 14:57:15.119486 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.119436 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/020f32d2-c323-430c-b6a6-d97a25182801-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-dntl4\" (UID: \"020f32d2-c323-430c-b6a6-d97a25182801\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-dntl4" Jul 24 14:57:15.121313 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.121277 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2872026a-6cfb-4c62-8ae5-b33297a9b1e7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-86bxn\" (UID: \"2872026a-6cfb-4c62-8ae5-b33297a9b1e7\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn" Jul 24 14:57:15.242933 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.242894 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-26tts\"" Jul 24 14:57:15.250935 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.250907 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:57:15.270821 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.270753 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-czcf5\"" Jul 24 14:57:15.278735 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.278708 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:57:15.298476 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.298442 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tq8dm\"" Jul 24 14:57:15.305940 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.305916 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525" Jul 24 14:57:15.305940 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.305930 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9bsmn\"" Jul 24 14:57:15.314407 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.314386 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn" Jul 24 14:57:15.320393 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.320375 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-v2kd8\"" Jul 24 14:57:15.329318 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.329295 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-dntl4" Jul 24 14:57:15.331819 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.331801 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bc4wp\"" Jul 24 14:57:15.339805 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.339783 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2jdtw" Jul 24 14:57:15.637582 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.637496 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mjqx9" event={"ID":"ec175d02-f2a5-4492-8133-e642a367da32","Type":"ContainerStarted","Data":"e2994c4e1083b7ad06ce3d84e2bc5e78bdad42bcbc0b83a12b98fea2b7292ed6"} Jul 24 14:57:15.638817 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.638787 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xgswn" event={"ID":"a9494b1d-f2bb-408b-8283-7d622ea5e450","Type":"ContainerStarted","Data":"8050d0d98a19d0e0c06e173cad9207d502145f25524d7edf428ec8a3ca1b6ca9"} Jul 24 14:57:15.971972 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.971754 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6jf8n"] Jul 24 14:57:15.982494 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:57:15.982047 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod441a2a3d_5df6_4a88_92cc_63d18ba4310f.slice/crio-aad6e2a7d3be4266c77bd64bfa9d65071f6c9bbc1a8896da36a187c404e1877f WatchSource:0}: Error finding container aad6e2a7d3be4266c77bd64bfa9d65071f6c9bbc1a8896da36a187c404e1877f: Status 404 returned error can't find the container with id aad6e2a7d3be4266c77bd64bfa9d65071f6c9bbc1a8896da36a187c404e1877f Jul 24 14:57:15.996453 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:15.996351 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-779758b8b8-dc8j5"] Jul 24 14:57:16.003447 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.001075 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-dntl4"] Jul 24 14:57:16.014831 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:57:16.009982 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod020f32d2_c323_430c_b6a6_d97a25182801.slice/crio-c48977dfdce3d1722c61af5d95e384711100d902a9b18c1da6002e7bc9b0b3f2 WatchSource:0}: Error finding container c48977dfdce3d1722c61af5d95e384711100d902a9b18c1da6002e7bc9b0b3f2: Status 404 returned error can't find the container with id c48977dfdce3d1722c61af5d95e384711100d902a9b18c1da6002e7bc9b0b3f2 Jul 24 14:57:16.014831 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.013982 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2jdtw"] Jul 24 14:57:16.018955 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.018092 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn"] Jul 24 14:57:16.024653 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:57:16.024024 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31f123ed_b090_4d80_a46b_888032edc051.slice/crio-fe9aff50754220b6fe210567eb16ae60f15d8a5e6783072cb5034525c5ceb689 WatchSource:0}: Error finding container fe9aff50754220b6fe210567eb16ae60f15d8a5e6783072cb5034525c5ceb689: Status 404 returned error can't find the container with id fe9aff50754220b6fe210567eb16ae60f15d8a5e6783072cb5034525c5ceb689 Jul 24 14:57:16.024845 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:57:16.024669 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2872026a_6cfb_4c62_8ae5_b33297a9b1e7.slice/crio-e712a4e3456e52a786ccfa182e5db4f76634315d5a17f1efac1892ac4de124d1 WatchSource:0}: Error finding container e712a4e3456e52a786ccfa182e5db4f76634315d5a17f1efac1892ac4de124d1: Status 404 returned error can't find the container with id e712a4e3456e52a786ccfa182e5db4f76634315d5a17f1efac1892ac4de124d1 Jul 24 14:57:16.053579 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.053542 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-776fc686df-t8g55"] Jul 24 14:57:16.074050 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.073919 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525"] Jul 24 14:57:16.654012 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.653891 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525" event={"ID":"afe11285-00c7-47a8-a0d0-08acd3df5bda","Type":"ContainerStarted","Data":"e1de3090c367da5411b68b34315dbfa2ce232680b1b504106ff43064473680fe"} Jul 24 14:57:16.655642 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.655610 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2jdtw" event={"ID":"31f123ed-b090-4d80-a46b-888032edc051","Type":"ContainerStarted","Data":"fe9aff50754220b6fe210567eb16ae60f15d8a5e6783072cb5034525c5ceb689"} Jul 24 14:57:16.659132 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.659104 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-dntl4" event={"ID":"020f32d2-c323-430c-b6a6-d97a25182801","Type":"ContainerStarted","Data":"c48977dfdce3d1722c61af5d95e384711100d902a9b18c1da6002e7bc9b0b3f2"} Jul 24 14:57:16.661414 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.661392 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p9x5t" event={"ID":"54da61a7-9d56-41ec-8536-b1e4bfe30fb3","Type":"ContainerStarted","Data":"5e010ea8b6a40ca135ab298767adec7ee6d1cf823734f8292932b791029ef3d4"} Jul 24 14:57:16.663754 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.663723 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mjqx9" event={"ID":"ec175d02-f2a5-4492-8133-e642a367da32","Type":"ContainerStarted","Data":"f232ebda6d1411c856baab024577957e11fdea80c46d6c6066859b82ceac4e5f"} Jul 24 14:57:16.664510 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.664480 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:57:16.666881 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.666844 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xgswn" event={"ID":"a9494b1d-f2bb-408b-8283-7d622ea5e450","Type":"ContainerStarted","Data":"14c4a7dcebe5c1a74d82ad1405ccb830cb52f335d2900097db2f379da9433e1e"} Jul 24 14:57:16.668922 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.668897 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn" event={"ID":"2872026a-6cfb-4c62-8ae5-b33297a9b1e7","Type":"ContainerStarted","Data":"e712a4e3456e52a786ccfa182e5db4f76634315d5a17f1efac1892ac4de124d1"} Jul 24 14:57:16.671010 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.670985 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-776fc686df-t8g55" event={"ID":"3ede17c9-5194-41b7-a4e6-55a258fe1a4e","Type":"ContainerStarted","Data":"88c3cb9fa54181a62929b7938506bc3fdb17ac0f33a138091fbb4c6812ef165b"} Jul 24 14:57:16.671102 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.671031 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-776fc686df-t8g55" event={"ID":"3ede17c9-5194-41b7-a4e6-55a258fe1a4e","Type":"ContainerStarted","Data":"255b786fee739c0829715c6d35449a100163cad1bcb976c725e508693dcdc0ac"} Jul 24 14:57:16.673435 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.673415 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" event={"ID":"888b99fe-7dd7-499b-9326-37b892a7ef56","Type":"ContainerStarted","Data":"678c1dbbaee2c60d04fa00b88a200e9fedc40654e7afd5e4a648e0e55bf35cb9"} Jul 24 14:57:16.673858 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.673716 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:57:16.674036 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.674003 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" event={"ID":"888b99fe-7dd7-499b-9326-37b892a7ef56","Type":"ContainerStarted","Data":"f31befdc781502d44878d7d94a9a5d3e30f4f0ccb6b7ba4932a5a94d74266a1a"} Jul 24 14:57:16.677311 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.677254 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6jf8n" event={"ID":"441a2a3d-5df6-4a88-92cc-63d18ba4310f","Type":"ContainerStarted","Data":"aad6e2a7d3be4266c77bd64bfa9d65071f6c9bbc1a8896da36a187c404e1877f"} Jul 24 14:57:16.684771 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.684414 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mjqx9" podStartSLOduration=67.684397136 podStartE2EDuration="1m7.684397136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:56:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:57:16.682727503 +0000 UTC m=+68.913204212" watchObservedRunningTime="2026-07-24 14:57:16.684397136 +0000 UTC m=+68.914873821" Jul 24 14:57:16.726014 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.724460 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-776fc686df-t8g55" podStartSLOduration=53.7244389 podStartE2EDuration="53.7244389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:56:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:57:16.72410537 +0000 UTC m=+68.954582083" watchObservedRunningTime="2026-07-24 14:57:16.7244389 +0000 UTC m=+68.954915562" Jul 24 14:57:16.726633 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:16.726592 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" podStartSLOduration=67.726580944 podStartE2EDuration="1m7.726580944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:56:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:57:16.704049682 +0000 UTC m=+68.934526389" watchObservedRunningTime="2026-07-24 14:57:16.726580944 +0000 UTC m=+68.957057629" Jul 24 14:57:17.251744 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:17.251713 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:57:17.255006 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:17.254718 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:57:17.689555 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:17.689488 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xgswn" event={"ID":"a9494b1d-f2bb-408b-8283-7d622ea5e450","Type":"ContainerStarted","Data":"ba43f5159cf8e0f89b07396eb40b96f9d5249b5b017f2ab762f233b4ad40407a"} Jul 24 14:57:17.689555 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:17.689553 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:57:17.691543 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:17.691518 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-776fc686df-t8g55" Jul 24 14:57:17.708407 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:17.708352 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xgswn" podStartSLOduration=67.273057852 podStartE2EDuration="1m8.708331532s" totalImagesPullingTime="1.43527368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:57:17.706970108 +0000 UTC m=+69.937446821" watchObservedRunningTime="2026-07-24 14:57:17.708331532 +0000 UTC m=+69.938808219" Jul 24 14:57:18.273799 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:18.273769 2531 scope.go:117] "RemoveContainer" containerID="48db6626fe5e6625f72db8c795a865191a0c2cf4f31a98a3501ce023e1c48630" Jul 24 14:57:20.514959 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:20.514842 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:20.711608 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:20.711567 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:20.816060 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:20.815995 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:22.536033 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:22.536001 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:22.605364 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:22.605285 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:22.945781 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:22.945749 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:23.018427 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:23.018401 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:23.115306 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:23.115276 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:23.178802 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:23.178773 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:24.597975 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.597814 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:24.697463 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.697359 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:24.716468 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.716438 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn" event={"ID":"2872026a-6cfb-4c62-8ae5-b33297a9b1e7","Type":"ContainerStarted","Data":"73d79507053d6f2e0b41dfa3e7126ad815de67417f829093b701822fac25652b"} Jul 24 14:57:24.718843 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.718787 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6jf8n" event={"ID":"441a2a3d-5df6-4a88-92cc-63d18ba4310f","Type":"ContainerStarted","Data":"93c47c9398d44637b60ad7db7f1df9a181376f0fb0ae0a27435d9c0a79dc490c"} Jul 24 14:57:24.721761 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.721730 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525" event={"ID":"afe11285-00c7-47a8-a0d0-08acd3df5bda","Type":"ContainerStarted","Data":"769deccbbd83d054868c139b524ac7433c64cd30aa39694e6466fb7804cb4653"} Jul 24 14:57:24.721971 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.721955 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525" event={"ID":"afe11285-00c7-47a8-a0d0-08acd3df5bda","Type":"ContainerStarted","Data":"81b3bc64b89a5e4d1577f3c8ad5f4a197774550cb948694e08e22c55afed5744"} Jul 24 14:57:24.724245 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.724219 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2jdtw" event={"ID":"31f123ed-b090-4d80-a46b-888032edc051","Type":"ContainerStarted","Data":"91be69ac4da77c5a9426c5efa831837d9ce7764c62b0c578b38489ad02a3b06b"} Jul 24 14:57:24.724361 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.724270 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2jdtw" event={"ID":"31f123ed-b090-4d80-a46b-888032edc051","Type":"ContainerStarted","Data":"c4b555abfc26fa32d2cecc9cdfa7dbb606619e8f3588c25bd08258bb6b185c6a"} Jul 24 14:57:24.724361 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.724300 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2jdtw" Jul 24 14:57:24.726688 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.726628 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-dntl4" event={"ID":"020f32d2-c323-430c-b6a6-d97a25182801","Type":"ContainerStarted","Data":"51f25b652c8a17a8b761c7441c04112dc7c31146d05442b14f02cf21aab1bada"} Jul 24 14:57:24.728785 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.728764 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 14:57:24.728912 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.728853 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" event={"ID":"520066e2-8b42-4872-b257-20df38f77e37","Type":"ContainerStarted","Data":"5d415ea69735873e2ee925e213041d2927554addefee52fb6e59cb6ef318db14"} Jul 24 14:57:24.729233 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.729198 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" Jul 24 14:57:24.730943 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.730921 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p9x5t" event={"ID":"54da61a7-9d56-41ec-8536-b1e4bfe30fb3","Type":"ContainerStarted","Data":"e73062ce78c94d1a81d659e3e7b6e26eda1484de4b12c16e43d67b4ee6f1d33f"} Jul 24 14:57:24.741373 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.741327 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-86bxn" podStartSLOduration=53.639541406 podStartE2EDuration="1m1.741315371s" totalImagesPullingTime="8.101773965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:57:24.739989807 +0000 UTC m=+76.970466557" watchObservedRunningTime="2026-07-24 14:57:24.741315371 +0000 UTC m=+76.971792033" Jul 24 14:57:24.757851 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.757762 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vl525" podStartSLOduration=53.747805828 podStartE2EDuration="1m1.757746116s" totalImagesPullingTime="8.009940288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:57:24.756151966 +0000 UTC m=+76.986628675" watchObservedRunningTime="2026-07-24 14:57:24.757746116 +0000 UTC m=+76.988222787" Jul 24 14:57:24.778847 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.778804 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6jf8n" podStartSLOduration=67.630166121 podStartE2EDuration="1m15.778792953s" totalImagesPullingTime="8.148626832s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:57:24.77703665 +0000 UTC m=+77.007513344" watchObservedRunningTime="2026-07-24 14:57:24.778792953 +0000 UTC m=+77.009269636" Jul 24 14:57:24.788751 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.788717 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:24.800077 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.800038 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2jdtw" podStartSLOduration=33.70184956 podStartE2EDuration="41.800026313s" totalImagesPullingTime="8.098176753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:57:24.799460244 +0000 UTC m=+77.029936951" watchObservedRunningTime="2026-07-24 14:57:24.800026313 +0000 UTC m=+77.030502996" Jul 24 14:57:24.820133 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.819547 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" podStartSLOduration=52.819912328 podStartE2EDuration="1m1.819537205s" totalImagesPullingTime="8.999624877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:57:24.819317501 +0000 UTC m=+77.049794185" watchObservedRunningTime="2026-07-24 14:57:24.819537205 +0000 UTC m=+77.050013890" Jul 24 14:57:24.852916 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.852850 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-dntl4" podStartSLOduration=49.736084634 podStartE2EDuration="57.852836091s" totalImagesPullingTime="8.116751457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:56:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:57:24.850471618 +0000 UTC m=+77.080948314" watchObservedRunningTime="2026-07-24 14:57:24.852836091 +0000 UTC m=+77.083312775" Jul 24 14:57:24.964122 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:24.964088 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:25.065191 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:25.065112 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:25.105160 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:25.105134 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-77k6k" Jul 24 14:57:25.167183 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:25.167149 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:25.308496 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:25.308464 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:25.382403 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:25.382262 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:25.487313 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:25.487268 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:25.591723 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:25.591670 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:25.783515 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:25.783483 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:25.972583 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:25.972547 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:26.037535 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:26.037445 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:26.202971 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:26.202929 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:26.270771 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:26.270730 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:26.986076 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:26.986038 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:27.199884 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:27.199827 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:27.335193 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:27.335090 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:28.209559 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:28.209526 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:28.276684 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:28.276651 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:28.752992 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:28.752960 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p9x5t" event={"ID":"54da61a7-9d56-41ec-8536-b1e4bfe30fb3","Type":"ContainerStarted","Data":"b9bedea54280aa200aa81ab30451670daee6419da530f1d8ff5bab9992dc9cc3"} Jul 24 14:57:28.774106 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:28.774057 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-p9x5t" podStartSLOduration=17.349329366 podStartE2EDuration="34.774041448s" totalImagesPullingTime="17.424712082s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:56:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:57:28.773010946 +0000 UTC m=+81.003487658" watchObservedRunningTime="2026-07-24 14:57:28.774041448 +0000 UTC m=+81.004518152" Jul 24 14:57:31.140978 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:31.140940 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:31.221417 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:31.221376 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:31.809878 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:31.809815 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:31.896148 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:31.896111 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:32.524017 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:32.523982 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:33.025813 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:33.025772 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:33.093572 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:33.093515 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:34.737627 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:34.737595 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2jdtw" Jul 24 14:57:34.848244 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:34.848208 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:34.938680 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:34.938628 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:37.038375 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.038320 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:37.272045 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.272010 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:37.413352 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.413263 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:37.550518 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.550490 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz"] Jul 24 14:57:37.553454 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.553429 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb"] Jul 24 14:57:37.553593 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.553575 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz" Jul 24 14:57:37.555960 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.555940 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" Jul 24 14:57:37.574843 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.574824 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 24 14:57:37.574981 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.574832 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 24 14:57:37.574981 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.574832 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 24 14:57:37.575650 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.575620 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-gxgzw\"" Jul 24 14:57:37.575650 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.575633 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 24 14:57:37.575766 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.575636 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 24 14:57:37.579318 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.579299 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-5cdz5\"" Jul 24 14:57:37.585519 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.585500 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 24 14:57:37.587426 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.587391 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb"] Jul 24 14:57:37.598402 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.598385 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz"] Jul 24 14:57:37.606763 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.606733 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:37.611650 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.611630 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2d4gx"] Jul 24 14:57:37.613970 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.613948 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.616539 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.616523 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 24 14:57:37.617112 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.617092 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 24 14:57:37.617433 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.617415 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 24 14:57:37.617651 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.617633 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-89sm5\"" Jul 24 14:57:37.694534 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.694513 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:57:37.702723 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.702695 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:37.705794 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.705769 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7fbb1524-c4a9-4478-b9ab-be53b6f81309-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.705880 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.705816 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7fbb1524-c4a9-4478-b9ab-be53b6f81309-node-exporter-wtmp\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.705880 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.705843 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6550c456-f2c1-4d34-8812-e14246b4aae0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-llnsz\" (UID: \"6550c456-f2c1-4d34-8812-e14246b4aae0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz" Jul 24 14:57:37.705880 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.705878 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7fbb1524-c4a9-4478-b9ab-be53b6f81309-node-exporter-textfile\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.706038 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.705948 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7fbb1524-c4a9-4478-b9ab-be53b6f81309-root\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.706038 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.705981 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7fbb1524-c4a9-4478-b9ab-be53b6f81309-node-exporter-tls\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.706143 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.706037 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n5g5\" (UniqueName: \"kubernetes.io/projected/98a7d25e-e0dc-4170-b75c-826ac325a54b-kube-api-access-9n5g5\") pod \"kube-state-metrics-56d9f944b4-7ljrb\" (UID: \"98a7d25e-e0dc-4170-b75c-826ac325a54b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" Jul 24 14:57:37.706143 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.706073 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6550c456-f2c1-4d34-8812-e14246b4aae0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-llnsz\" (UID: \"6550c456-f2c1-4d34-8812-e14246b4aae0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz" Jul 24 14:57:37.706143 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.706101 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj99x\" (UniqueName: \"kubernetes.io/projected/6550c456-f2c1-4d34-8812-e14246b4aae0-kube-api-access-lj99x\") pod \"openshift-state-metrics-b8f49f5c6-llnsz\" (UID: \"6550c456-f2c1-4d34-8812-e14246b4aae0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz" Jul 24 14:57:37.706143 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.706129 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vngxc\" (UniqueName: \"kubernetes.io/projected/7fbb1524-c4a9-4478-b9ab-be53b6f81309-kube-api-access-vngxc\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.706276 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.706156 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7fbb1524-c4a9-4478-b9ab-be53b6f81309-metrics-client-ca\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.706276 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.706182 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6550c456-f2c1-4d34-8812-e14246b4aae0-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-llnsz\" (UID: \"6550c456-f2c1-4d34-8812-e14246b4aae0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz" Jul 24 14:57:37.706276 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.706206 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98a7d25e-e0dc-4170-b75c-826ac325a54b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-7ljrb\" (UID: \"98a7d25e-e0dc-4170-b75c-826ac325a54b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" Jul 24 14:57:37.706276 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.706232 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/98a7d25e-e0dc-4170-b75c-826ac325a54b-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-7ljrb\" (UID: \"98a7d25e-e0dc-4170-b75c-826ac325a54b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" Jul 24 14:57:37.706276 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.706261 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/98a7d25e-e0dc-4170-b75c-826ac325a54b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-7ljrb\" (UID: \"98a7d25e-e0dc-4170-b75c-826ac325a54b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" Jul 24 14:57:37.706433 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.706282 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7fbb1524-c4a9-4478-b9ab-be53b6f81309-sys\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.706433 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.706306 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/98a7d25e-e0dc-4170-b75c-826ac325a54b-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-7ljrb\" (UID: \"98a7d25e-e0dc-4170-b75c-826ac325a54b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" Jul 24 14:57:37.706433 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.706324 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98a7d25e-e0dc-4170-b75c-826ac325a54b-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-7ljrb\" (UID: \"98a7d25e-e0dc-4170-b75c-826ac325a54b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" Jul 24 14:57:37.706433 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.706342 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7fbb1524-c4a9-4478-b9ab-be53b6f81309-node-exporter-accelerators-collector-config\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.806712 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.806680 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7fbb1524-c4a9-4478-b9ab-be53b6f81309-node-exporter-wtmp\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.806712 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.806709 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:37.806984 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.806751 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6550c456-f2c1-4d34-8812-e14246b4aae0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-llnsz\" (UID: \"6550c456-f2c1-4d34-8812-e14246b4aae0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz" Jul 24 14:57:37.806984 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.806783 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7fbb1524-c4a9-4478-b9ab-be53b6f81309-node-exporter-textfile\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.806984 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.806848 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7fbb1524-c4a9-4478-b9ab-be53b6f81309-root\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.806984 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.806879 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7fbb1524-c4a9-4478-b9ab-be53b6f81309-node-exporter-wtmp\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.806984 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.806926 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7fbb1524-c4a9-4478-b9ab-be53b6f81309-root\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.806984 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.806961 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7fbb1524-c4a9-4478-b9ab-be53b6f81309-node-exporter-tls\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.807256 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.806987 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n5g5\" (UniqueName: \"kubernetes.io/projected/98a7d25e-e0dc-4170-b75c-826ac325a54b-kube-api-access-9n5g5\") pod \"kube-state-metrics-56d9f944b4-7ljrb\" (UID: \"98a7d25e-e0dc-4170-b75c-826ac325a54b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" Jul 24 14:57:37.807256 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.807019 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6550c456-f2c1-4d34-8812-e14246b4aae0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-llnsz\" (UID: \"6550c456-f2c1-4d34-8812-e14246b4aae0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz" Jul 24 14:57:37.807482 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.807454 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7fbb1524-c4a9-4478-b9ab-be53b6f81309-node-exporter-textfile\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.807611 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:57:37.807535 2531 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 24 14:57:37.807611 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.807557 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj99x\" (UniqueName: \"kubernetes.io/projected/6550c456-f2c1-4d34-8812-e14246b4aae0-kube-api-access-lj99x\") pod \"openshift-state-metrics-b8f49f5c6-llnsz\" (UID: \"6550c456-f2c1-4d34-8812-e14246b4aae0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz" Jul 24 14:57:37.808122 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:57:37.808101 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fbb1524-c4a9-4478-b9ab-be53b6f81309-node-exporter-tls podName:7fbb1524-c4a9-4478-b9ab-be53b6f81309 nodeName:}" failed. No retries permitted until 2026-07-24 14:57:38.308077496 +0000 UTC m=+90.538554158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/7fbb1524-c4a9-4478-b9ab-be53b6f81309-node-exporter-tls") pod "node-exporter-2d4gx" (UID: "7fbb1524-c4a9-4478-b9ab-be53b6f81309") : secret "node-exporter-tls" not found Jul 24 14:57:37.808302 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.808284 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vngxc\" (UniqueName: \"kubernetes.io/projected/7fbb1524-c4a9-4478-b9ab-be53b6f81309-kube-api-access-vngxc\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.808390 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.808372 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7fbb1524-c4a9-4478-b9ab-be53b6f81309-metrics-client-ca\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.808449 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.808404 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6550c456-f2c1-4d34-8812-e14246b4aae0-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-llnsz\" (UID: \"6550c456-f2c1-4d34-8812-e14246b4aae0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz" Jul 24 14:57:37.808504 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.808444 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98a7d25e-e0dc-4170-b75c-826ac325a54b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-7ljrb\" (UID: \"98a7d25e-e0dc-4170-b75c-826ac325a54b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" Jul 24 14:57:37.808504 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.808482 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/98a7d25e-e0dc-4170-b75c-826ac325a54b-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-7ljrb\" (UID: \"98a7d25e-e0dc-4170-b75c-826ac325a54b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" Jul 24 14:57:37.808598 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.808518 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/98a7d25e-e0dc-4170-b75c-826ac325a54b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-7ljrb\" (UID: \"98a7d25e-e0dc-4170-b75c-826ac325a54b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" Jul 24 14:57:37.808598 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.808551 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7fbb1524-c4a9-4478-b9ab-be53b6f81309-sys\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.808736 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.808635 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/98a7d25e-e0dc-4170-b75c-826ac325a54b-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-7ljrb\" (UID: \"98a7d25e-e0dc-4170-b75c-826ac325a54b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" Jul 24 14:57:37.809531 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.809507 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98a7d25e-e0dc-4170-b75c-826ac325a54b-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-7ljrb\" (UID: \"98a7d25e-e0dc-4170-b75c-826ac325a54b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" Jul 24 14:57:37.812722 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.809562 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7fbb1524-c4a9-4478-b9ab-be53b6f81309-node-exporter-accelerators-collector-config\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.812722 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.809619 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7fbb1524-c4a9-4478-b9ab-be53b6f81309-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.812722 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.809728 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7fbb1524-c4a9-4478-b9ab-be53b6f81309-metrics-client-ca\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.812722 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.810849 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6550c456-f2c1-4d34-8812-e14246b4aae0-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-llnsz\" (UID: \"6550c456-f2c1-4d34-8812-e14246b4aae0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz" Jul 24 14:57:37.812722 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.810961 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7fbb1524-c4a9-4478-b9ab-be53b6f81309-sys\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.812722 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.811322 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98a7d25e-e0dc-4170-b75c-826ac325a54b-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-7ljrb\" (UID: \"98a7d25e-e0dc-4170-b75c-826ac325a54b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" Jul 24 14:57:37.812722 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.811324 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/98a7d25e-e0dc-4170-b75c-826ac325a54b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-7ljrb\" (UID: \"98a7d25e-e0dc-4170-b75c-826ac325a54b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" Jul 24 14:57:37.812722 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.812341 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6550c456-f2c1-4d34-8812-e14246b4aae0-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-llnsz\" (UID: \"6550c456-f2c1-4d34-8812-e14246b4aae0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz" Jul 24 14:57:37.812722 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.812590 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/98a7d25e-e0dc-4170-b75c-826ac325a54b-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-7ljrb\" (UID: \"98a7d25e-e0dc-4170-b75c-826ac325a54b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" Jul 24 14:57:37.813249 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.813030 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7fbb1524-c4a9-4478-b9ab-be53b6f81309-node-exporter-accelerators-collector-config\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.814586 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.813981 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7fbb1524-c4a9-4478-b9ab-be53b6f81309-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.814586 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.814053 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98a7d25e-e0dc-4170-b75c-826ac325a54b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-7ljrb\" (UID: \"98a7d25e-e0dc-4170-b75c-826ac325a54b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" Jul 24 14:57:37.814983 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.814955 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/98a7d25e-e0dc-4170-b75c-826ac325a54b-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-7ljrb\" (UID: \"98a7d25e-e0dc-4170-b75c-826ac325a54b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" Jul 24 14:57:37.815661 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.815642 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6550c456-f2c1-4d34-8812-e14246b4aae0-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-llnsz\" (UID: \"6550c456-f2c1-4d34-8812-e14246b4aae0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz" Jul 24 14:57:37.823429 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.823411 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n5g5\" (UniqueName: \"kubernetes.io/projected/98a7d25e-e0dc-4170-b75c-826ac325a54b-kube-api-access-9n5g5\") pod \"kube-state-metrics-56d9f944b4-7ljrb\" (UID: \"98a7d25e-e0dc-4170-b75c-826ac325a54b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" Jul 24 14:57:37.823831 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.823810 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj99x\" (UniqueName: \"kubernetes.io/projected/6550c456-f2c1-4d34-8812-e14246b4aae0-kube-api-access-lj99x\") pod \"openshift-state-metrics-b8f49f5c6-llnsz\" (UID: \"6550c456-f2c1-4d34-8812-e14246b4aae0\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz" Jul 24 14:57:37.824104 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.824088 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vngxc\" (UniqueName: \"kubernetes.io/projected/7fbb1524-c4a9-4478-b9ab-be53b6f81309-kube-api-access-vngxc\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:37.863595 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.863574 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz" Jul 24 14:57:37.868164 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.868149 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" Jul 24 14:57:37.904893 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:37.904835 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:38.008069 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.008040 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz"] Jul 24 14:57:38.010388 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:57:38.010361 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6550c456_f2c1_4d34_8812_e14246b4aae0.slice/crio-0e97fb1d9e66652c067aa3f6141528f1f0a995161285755cfc58dd43746610c0 WatchSource:0}: Error finding container 0e97fb1d9e66652c067aa3f6141528f1f0a995161285755cfc58dd43746610c0: Status 404 returned error can't find the container with id 0e97fb1d9e66652c067aa3f6141528f1f0a995161285755cfc58dd43746610c0 Jul 24 14:57:38.020451 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.020428 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb"] Jul 24 14:57:38.023559 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:57:38.023533 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98a7d25e_e0dc_4170_b75c_826ac325a54b.slice/crio-462c826c71b8cb3984093da5ed7b3e296731fa1ffbcc3006b4d50fa1f78c007e WatchSource:0}: Error finding container 462c826c71b8cb3984093da5ed7b3e296731fa1ffbcc3006b4d50fa1f78c007e: Status 404 returned error can't find the container with id 462c826c71b8cb3984093da5ed7b3e296731fa1ffbcc3006b4d50fa1f78c007e Jul 24 14:57:38.100083 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.100047 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:38.231817 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.231740 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:38.316843 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.316809 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7fbb1524-c4a9-4478-b9ab-be53b6f81309-node-exporter-tls\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:38.319172 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.319156 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7fbb1524-c4a9-4478-b9ab-be53b6f81309-node-exporter-tls\") pod \"node-exporter-2d4gx\" (UID: \"7fbb1524-c4a9-4478-b9ab-be53b6f81309\") " pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:38.334241 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.334216 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:38.368471 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.368441 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 14:57:38.371174 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.371160 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.374505 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.374481 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 24 14:57:38.374597 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.374572 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 24 14:57:38.374660 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.374609 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 24 14:57:38.374809 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.374794 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 24 14:57:38.374905 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.374808 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 24 14:57:38.374971 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.374924 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 24 14:57:38.375086 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.375070 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 24 14:57:38.375146 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.375082 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 24 14:57:38.375146 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.375082 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bhgsl\"" Jul 24 14:57:38.375394 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.375382 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 24 14:57:38.395076 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.395051 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 14:57:38.450170 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.450138 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:38.518630 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.518549 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7976279e-37d6-43f2-9f99-be47eb125d4f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.518630 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.518595 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7976279e-37d6-43f2-9f99-be47eb125d4f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.519164 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.518636 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-config-volume\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.519164 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.518665 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-web-config\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.519164 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.518740 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.519164 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.518768 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.519164 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.518900 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.519164 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.518940 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw5b5\" (UniqueName: \"kubernetes.io/projected/7976279e-37d6-43f2-9f99-be47eb125d4f-kube-api-access-sw5b5\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.519164 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.518966 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.519164 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.519040 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7976279e-37d6-43f2-9f99-be47eb125d4f-config-out\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.519164 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.519058 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.519164 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.519078 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7976279e-37d6-43f2-9f99-be47eb125d4f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.519164 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.519127 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7976279e-37d6-43f2-9f99-be47eb125d4f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.523637 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.523613 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2d4gx" Jul 24 14:57:38.534031 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:57:38.533995 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fbb1524_c4a9_4478_b9ab_be53b6f81309.slice/crio-3702aaf780b607ba0192eac6eadc838b355bd660a26d5c9d482ffcda7a43878c WatchSource:0}: Error finding container 3702aaf780b607ba0192eac6eadc838b355bd660a26d5c9d482ffcda7a43878c: Status 404 returned error can't find the container with id 3702aaf780b607ba0192eac6eadc838b355bd660a26d5c9d482ffcda7a43878c Jul 24 14:57:38.556084 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.556056 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:38.619836 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.619805 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-config-volume\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.619975 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.619847 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-web-config\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.619975 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.619892 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.619975 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.619921 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.619975 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.619959 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.620142 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.619986 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw5b5\" (UniqueName: \"kubernetes.io/projected/7976279e-37d6-43f2-9f99-be47eb125d4f-kube-api-access-sw5b5\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.620142 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.620010 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.620142 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.620086 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7976279e-37d6-43f2-9f99-be47eb125d4f-config-out\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.620142 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.620109 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.620142 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.620136 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7976279e-37d6-43f2-9f99-be47eb125d4f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.620334 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.620158 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7976279e-37d6-43f2-9f99-be47eb125d4f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.620334 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.620193 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7976279e-37d6-43f2-9f99-be47eb125d4f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.620334 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.620223 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7976279e-37d6-43f2-9f99-be47eb125d4f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.620982 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.620951 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7976279e-37d6-43f2-9f99-be47eb125d4f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.622424 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.622400 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7976279e-37d6-43f2-9f99-be47eb125d4f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.623264 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.623240 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7976279e-37d6-43f2-9f99-be47eb125d4f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.625003 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.624980 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-web-config\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.625765 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.625689 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.625947 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.625924 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7976279e-37d6-43f2-9f99-be47eb125d4f-config-out\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.626634 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.626258 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.626634 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.626389 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7976279e-37d6-43f2-9f99-be47eb125d4f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.626634 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.626574 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-config-volume\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.626634 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.626596 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.627185 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.627164 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.628091 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.628073 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.632524 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.632499 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw5b5\" (UniqueName: \"kubernetes.io/projected/7976279e-37d6-43f2-9f99-be47eb125d4f-kube-api-access-sw5b5\") pod \"alertmanager-main-0\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.670891 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.670852 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:38.680775 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.680750 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:57:38.761839 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.761807 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:38.791759 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.791715 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2d4gx" event={"ID":"7fbb1524-c4a9-4478-b9ab-be53b6f81309","Type":"ContainerStarted","Data":"3702aaf780b607ba0192eac6eadc838b355bd660a26d5c9d482ffcda7a43878c"} Jul 24 14:57:38.793704 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.793676 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" event={"ID":"98a7d25e-e0dc-4170-b75c-826ac325a54b","Type":"ContainerStarted","Data":"462c826c71b8cb3984093da5ed7b3e296731fa1ffbcc3006b4d50fa1f78c007e"} Jul 24 14:57:38.797410 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.797199 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz" event={"ID":"6550c456-f2c1-4d34-8812-e14246b4aae0","Type":"ContainerStarted","Data":"ad4f438bd8f8d93bce982ba1249e3d554e28f351a36b548be3fc300eec7ccb30"} Jul 24 14:57:38.797410 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.797261 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz" event={"ID":"6550c456-f2c1-4d34-8812-e14246b4aae0","Type":"ContainerStarted","Data":"0d8dafbbe31f26aa8a7d54d8e21a2c917b422f90ed3a3a9d853267d2a4b56151"} Jul 24 14:57:38.797410 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.797293 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz" event={"ID":"6550c456-f2c1-4d34-8812-e14246b4aae0","Type":"ContainerStarted","Data":"0e97fb1d9e66652c067aa3f6141528f1f0a995161285755cfc58dd43746610c0"} Jul 24 14:57:38.851513 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:38.851416 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 14:57:38.855693 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:57:38.855658 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7976279e_37d6_43f2_9f99_be47eb125d4f.slice/crio-fa4a15cc4db219d56d6820f21b94e3c25fbf058d54df345e9a19d3a10edec12f WatchSource:0}: Error finding container fa4a15cc4db219d56d6820f21b94e3c25fbf058d54df345e9a19d3a10edec12f: Status 404 returned error can't find the container with id fa4a15cc4db219d56d6820f21b94e3c25fbf058d54df345e9a19d3a10edec12f Jul 24 14:57:39.606432 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:39.605802 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:39.806035 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:39.805958 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" event={"ID":"98a7d25e-e0dc-4170-b75c-826ac325a54b","Type":"ContainerStarted","Data":"9c6f881d6a8d7be77169e84282fb98ee076c5288c0fb3c5c196969c812bc5161"} Jul 24 14:57:39.806195 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:39.806058 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" event={"ID":"98a7d25e-e0dc-4170-b75c-826ac325a54b","Type":"ContainerStarted","Data":"7c28c0e41dc2114169130024573127bff018c9ee4741761ce02f408fd3525d43"} Jul 24 14:57:39.807553 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:39.807528 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7976279e-37d6-43f2-9f99-be47eb125d4f","Type":"ContainerStarted","Data":"fa4a15cc4db219d56d6820f21b94e3c25fbf058d54df345e9a19d3a10edec12f"} Jul 24 14:57:39.810376 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:39.810350 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz" event={"ID":"6550c456-f2c1-4d34-8812-e14246b4aae0","Type":"ContainerStarted","Data":"4400b018a0f75a6c2ac5af334ee2c7b5f1d38befeea395646bd18b93f2fa74a1"} Jul 24 14:57:39.840883 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:39.840027 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-llnsz" podStartSLOduration=1.446117234 podStartE2EDuration="2.840009057s" totalImagesPullingTime="1.393891823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:57:39.836302811 +0000 UTC m=+92.066779512" watchObservedRunningTime="2026-07-24 14:57:39.840009057 +0000 UTC m=+92.070485743" Jul 24 14:57:40.129320 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:40.129168 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:40.224494 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:40.224448 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:40.367552 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:40.367519 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:40.441207 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:40.441172 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:40.540448 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:40.540418 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:40.729536 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:40.729509 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:40.816183 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:40.816148 2531 generic.go:356] "Generic (PLEG): container finished" podID="7fbb1524-c4a9-4478-b9ab-be53b6f81309" containerID="1c5bd765412dcb3e9e09015152212cd45e29a9e6b464040865ee3128c6f57532" exitCode=0 Jul 24 14:57:40.816329 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:40.816230 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2d4gx" event={"ID":"7fbb1524-c4a9-4478-b9ab-be53b6f81309","Type":"ContainerDied","Data":"1c5bd765412dcb3e9e09015152212cd45e29a9e6b464040865ee3128c6f57532"} Jul 24 14:57:40.818956 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:40.818931 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" event={"ID":"98a7d25e-e0dc-4170-b75c-826ac325a54b","Type":"ContainerStarted","Data":"b1c2672c835f2d7eb0aca1611c6007e0e07fe77f4a4dc743cbe6adb5b2534670"} Jul 24 14:57:40.821051 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:40.820924 2531 generic.go:356] "Generic (PLEG): container finished" podID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerID="3eaf70aa34fd7fd636209979fca53e2b2d5b37bd9d0df1988e7c0101d4617fc8" exitCode=0 Jul 24 14:57:40.821051 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:40.821025 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7976279e-37d6-43f2-9f99-be47eb125d4f","Type":"ContainerDied","Data":"3eaf70aa34fd7fd636209979fca53e2b2d5b37bd9d0df1988e7c0101d4617fc8"} Jul 24 14:57:40.856285 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:40.856241 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7ljrb" podStartSLOduration=2.3738746600000002 podStartE2EDuration="3.856226912s" totalImagesPullingTime="1.482352252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:57:40.854659256 +0000 UTC m=+93.085135965" watchObservedRunningTime="2026-07-24 14:57:40.856226912 +0000 UTC m=+93.086703594" Jul 24 14:57:41.827370 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:41.827332 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2d4gx" event={"ID":"7fbb1524-c4a9-4478-b9ab-be53b6f81309","Type":"ContainerStarted","Data":"2af468684d904a471eb855cc7bf671212ba7a7eb88b683049dfa80c123b81919"} Jul 24 14:57:41.827755 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:41.827419 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2d4gx" event={"ID":"7fbb1524-c4a9-4478-b9ab-be53b6f81309","Type":"ContainerStarted","Data":"f20f23b99f0b11ebb5c327e132e40aff2ac5077efd06ee9f174f277c560cbd7e"} Jul 24 14:57:41.857431 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:41.857374 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2d4gx" podStartSLOduration=3.356500637 podStartE2EDuration="4.85735519s" totalImagesPullingTime="1.42064926s" totalInitContainerRuntime="80.205293ms" isStatefulPod=true podCreationTimestamp="2026-07-24 14:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:57:41.85592581 +0000 UTC m=+94.086402522" watchObservedRunningTime="2026-07-24 14:57:41.85735519 +0000 UTC m=+94.087831875" Jul 24 14:57:42.021529 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.021468 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:42.036530 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.036499 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-gc7xf"] Jul 24 14:57:42.039065 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.039043 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gc7xf" Jul 24 14:57:42.044131 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.044103 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-28dmd\"" Jul 24 14:57:42.044247 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.044142 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 24 14:57:42.051370 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.051348 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/32e69dbd-71cf-4709-99e0-32c3db1d9a74-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-gc7xf\" (UID: \"32e69dbd-71cf-4709-99e0-32c3db1d9a74\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gc7xf" Jul 24 14:57:42.051490 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.051384 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxbdc\" (UniqueName: \"kubernetes.io/projected/32e69dbd-71cf-4709-99e0-32c3db1d9a74-kube-api-access-xxbdc\") pod \"monitoring-plugin-68b7fc9574-gc7xf\" (UID: \"32e69dbd-71cf-4709-99e0-32c3db1d9a74\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gc7xf" Jul 24 14:57:42.055191 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.055171 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-gc7xf"] Jul 24 14:57:42.099462 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.099353 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:42.152734 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.152699 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/32e69dbd-71cf-4709-99e0-32c3db1d9a74-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-gc7xf\" (UID: \"32e69dbd-71cf-4709-99e0-32c3db1d9a74\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gc7xf" Jul 24 14:57:42.152734 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.152737 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxbdc\" (UniqueName: \"kubernetes.io/projected/32e69dbd-71cf-4709-99e0-32c3db1d9a74-kube-api-access-xxbdc\") pod \"monitoring-plugin-68b7fc9574-gc7xf\" (UID: \"32e69dbd-71cf-4709-99e0-32c3db1d9a74\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gc7xf" Jul 24 14:57:42.152986 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:57:42.152849 2531 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 24 14:57:42.152986 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:57:42.152938 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e69dbd-71cf-4709-99e0-32c3db1d9a74-monitoring-plugin-cert podName:32e69dbd-71cf-4709-99e0-32c3db1d9a74 nodeName:}" failed. No retries permitted until 2026-07-24 14:57:42.652917802 +0000 UTC m=+94.883394465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/32e69dbd-71cf-4709-99e0-32c3db1d9a74-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-gc7xf" (UID: "32e69dbd-71cf-4709-99e0-32c3db1d9a74") : secret "monitoring-plugin-cert" not found Jul 24 14:57:42.162146 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.162124 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxbdc\" (UniqueName: \"kubernetes.io/projected/32e69dbd-71cf-4709-99e0-32c3db1d9a74-kube-api-access-xxbdc\") pod \"monitoring-plugin-68b7fc9574-gc7xf\" (UID: \"32e69dbd-71cf-4709-99e0-32c3db1d9a74\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gc7xf" Jul 24 14:57:42.475199 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.475169 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-779758b8b8-dc8j5"] Jul 24 14:57:42.598604 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.598567 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:42.658262 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.658232 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/32e69dbd-71cf-4709-99e0-32c3db1d9a74-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-gc7xf\" (UID: \"32e69dbd-71cf-4709-99e0-32c3db1d9a74\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gc7xf" Jul 24 14:57:42.658485 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:57:42.658389 2531 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 24 14:57:42.658485 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:57:42.658462 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e69dbd-71cf-4709-99e0-32c3db1d9a74-monitoring-plugin-cert podName:32e69dbd-71cf-4709-99e0-32c3db1d9a74 nodeName:}" failed. No retries permitted until 2026-07-24 14:57:43.658438516 +0000 UTC m=+95.888915199 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/32e69dbd-71cf-4709-99e0-32c3db1d9a74-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-gc7xf" (UID: "32e69dbd-71cf-4709-99e0-32c3db1d9a74") : secret "monitoring-plugin-cert" not found Jul 24 14:57:42.701857 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.701828 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:42.834241 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.834210 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7976279e-37d6-43f2-9f99-be47eb125d4f","Type":"ContainerStarted","Data":"ae2fc75777b1c6c263b7e4df23eaa5e893d3e78b77d09ab814165b7a49e37863"} Jul 24 14:57:42.834547 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.834275 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7976279e-37d6-43f2-9f99-be47eb125d4f","Type":"ContainerStarted","Data":"d40db5d184bcaf5632770c378fdb92ba178eada086b798ab7cfaf495fa177f81"} Jul 24 14:57:42.834547 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.834302 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7976279e-37d6-43f2-9f99-be47eb125d4f","Type":"ContainerStarted","Data":"2ffde93f0d416d77ccd16359f31468d5a0e26c272691fd8a45514b84db7f4e08"} Jul 24 14:57:42.834547 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.834325 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7976279e-37d6-43f2-9f99-be47eb125d4f","Type":"ContainerStarted","Data":"38b22758f176b9b3566ced6043a003cabf1c84dc65f4e50fe5c4a439495e5eea"} Jul 24 14:57:42.834547 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.834347 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7976279e-37d6-43f2-9f99-be47eb125d4f","Type":"ContainerStarted","Data":"a316ac02ad850ab0dc24f44cecd645e2f6175e66db50fae57d9647597d3f5324"} Jul 24 14:57:42.860837 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:42.860807 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:43.008570 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:43.008477 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:43.078630 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:43.078598 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:43.667401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:43.667368 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/32e69dbd-71cf-4709-99e0-32c3db1d9a74-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-gc7xf\" (UID: \"32e69dbd-71cf-4709-99e0-32c3db1d9a74\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gc7xf" Jul 24 14:57:43.670119 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:43.670090 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/32e69dbd-71cf-4709-99e0-32c3db1d9a74-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-gc7xf\" (UID: \"32e69dbd-71cf-4709-99e0-32c3db1d9a74\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gc7xf" Jul 24 14:57:43.681693 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:43.681665 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:43.840814 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:43.840762 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7976279e-37d6-43f2-9f99-be47eb125d4f","Type":"ContainerStarted","Data":"e8e3d44a3befc34fd6170e2cb3e31151ad0b2bfe6f18c54f3e801417e36a3b84"} Jul 24 14:57:43.849467 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:43.849451 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gc7xf" Jul 24 14:57:43.876617 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:43.876549 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.82425566 podStartE2EDuration="5.876529101s" totalImagesPullingTime="4.03565254s" totalInitContainerRuntime="16.620901ms" isStatefulPod=false podCreationTimestamp="2026-07-24 14:57:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:57:43.874491516 +0000 UTC m=+96.104968261" watchObservedRunningTime="2026-07-24 14:57:43.876529101 +0000 UTC m=+96.107005803" Jul 24 14:57:43.988583 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:43.988558 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-gc7xf"] Jul 24 14:57:43.991059 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:57:43.991027 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32e69dbd_71cf_4709_99e0_32c3db1d9a74.slice/crio-787624efb77d988692fb8c41f1037a175dc4167a8487e4dacde77a4009691220 WatchSource:0}: Error finding container 787624efb77d988692fb8c41f1037a175dc4167a8487e4dacde77a4009691220: Status 404 returned error can't find the container with id 787624efb77d988692fb8c41f1037a175dc4167a8487e4dacde77a4009691220 Jul 24 14:57:44.077740 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:44.077664 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:44.172782 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:44.172752 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:44.612794 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:44.612756 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:44.691194 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:44.691161 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:44.847239 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:44.847204 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gc7xf" event={"ID":"32e69dbd-71cf-4709-99e0-32c3db1d9a74","Type":"ContainerStarted","Data":"787624efb77d988692fb8c41f1037a175dc4167a8487e4dacde77a4009691220"} Jul 24 14:57:46.856938 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:46.856902 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gc7xf" event={"ID":"32e69dbd-71cf-4709-99e0-32c3db1d9a74","Type":"ContainerStarted","Data":"77bd9f137ff3427028ed5d12896ac298b1d0dcf63cb996bbfc6c80d292a48b7f"} Jul 24 14:57:46.857406 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:46.857102 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gc7xf" Jul 24 14:57:46.862049 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:46.862028 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gc7xf" Jul 24 14:57:46.875141 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:46.875102 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-gc7xf" podStartSLOduration=3.0813409 podStartE2EDuration="4.875088483s" totalImagesPullingTime="1.793747583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:57:46.874186679 +0000 UTC m=+99.104663386" watchObservedRunningTime="2026-07-24 14:57:46.875088483 +0000 UTC m=+99.105565168" Jul 24 14:57:47.174071 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:47.174015 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:47.248033 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:47.247990 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:47.700144 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:47.700004 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:47.763172 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:47.763144 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:48.459644 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.459615 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86bdb6974-zwqhn"] Jul 24 14:57:48.462990 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.462974 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.465383 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.465357 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 24 14:57:48.465383 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.465373 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 24 14:57:48.466193 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.466176 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gfgt9\"" Jul 24 14:57:48.466641 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.466624 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 24 14:57:48.466856 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.466843 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 24 14:57:48.466970 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.466914 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 24 14:57:48.466970 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.466918 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 24 14:57:48.470247 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.470125 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 24 14:57:48.472368 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.472352 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 24 14:57:48.475374 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.475352 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86bdb6974-zwqhn"] Jul 24 14:57:48.510879 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.510840 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-service-ca\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.510993 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.510900 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-oauth-serving-cert\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.510993 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.510938 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9aec1fce-7090-4a2e-a40a-4d087bf54f83-console-oauth-config\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.510993 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.510954 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvhtn\" (UniqueName: \"kubernetes.io/projected/9aec1fce-7090-4a2e-a40a-4d087bf54f83-kube-api-access-rvhtn\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.510993 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.510974 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-trusted-ca-bundle\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.511144 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.511033 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-console-config\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.511144 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.511106 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9aec1fce-7090-4a2e-a40a-4d087bf54f83-console-serving-cert\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.612326 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.612293 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9aec1fce-7090-4a2e-a40a-4d087bf54f83-console-serving-cert\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.612474 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.612336 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-service-ca\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.612474 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.612358 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-oauth-serving-cert\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.612556 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.612537 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9aec1fce-7090-4a2e-a40a-4d087bf54f83-console-oauth-config\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.612596 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.612573 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvhtn\" (UniqueName: \"kubernetes.io/projected/9aec1fce-7090-4a2e-a40a-4d087bf54f83-kube-api-access-rvhtn\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.612647 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.612616 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-trusted-ca-bundle\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.612763 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.612744 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-console-config\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.613186 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.613159 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-oauth-serving-cert\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.613454 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.613311 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-console-config\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.613454 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.613402 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-trusted-ca-bundle\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.613454 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.613419 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-service-ca\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.615021 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.614993 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9aec1fce-7090-4a2e-a40a-4d087bf54f83-console-serving-cert\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.615116 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.615046 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9aec1fce-7090-4a2e-a40a-4d087bf54f83-console-oauth-config\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.628893 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.628852 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvhtn\" (UniqueName: \"kubernetes.io/projected/9aec1fce-7090-4a2e-a40a-4d087bf54f83-kube-api-access-rvhtn\") pod \"console-86bdb6974-zwqhn\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.695365 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.695344 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mjqx9" Jul 24 14:57:48.773509 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.773442 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:48.820689 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.820645 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:48.895304 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.895264 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:48.921888 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:48.921808 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86bdb6974-zwqhn"] Jul 24 14:57:48.923917 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:57:48.923892 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aec1fce_7090_4a2e_a40a_4d087bf54f83.slice/crio-feb5d829562fc7ebb5cbb1271b6d9eb43270e79e3a506b9db822c5a0e6f94376 WatchSource:0}: Error finding container feb5d829562fc7ebb5cbb1271b6d9eb43270e79e3a506b9db822c5a0e6f94376: Status 404 returned error can't find the container with id feb5d829562fc7ebb5cbb1271b6d9eb43270e79e3a506b9db822c5a0e6f94376 Jul 24 14:57:49.872576 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:49.872538 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86bdb6974-zwqhn" event={"ID":"9aec1fce-7090-4a2e-a40a-4d087bf54f83","Type":"ContainerStarted","Data":"feb5d829562fc7ebb5cbb1271b6d9eb43270e79e3a506b9db822c5a0e6f94376"} Jul 24 14:57:50.419127 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:50.419081 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:51.884846 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:51.884813 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86bdb6974-zwqhn" event={"ID":"9aec1fce-7090-4a2e-a40a-4d087bf54f83","Type":"ContainerStarted","Data":"d415ebb88d30000a8fa859c6819ea5b0cf704ac53d0b36e891e51d855331371c"} Jul 24 14:57:51.905171 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:51.905121 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86bdb6974-zwqhn" podStartSLOduration=1.0970172360000001 podStartE2EDuration="3.905104831s" totalImagesPullingTime="2.808087595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:57:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:57:51.904080565 +0000 UTC m=+104.134557262" watchObservedRunningTime="2026-07-24 14:57:51.905104831 +0000 UTC m=+104.135581514" Jul 24 14:57:55.049289 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:55.049231 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:55.145169 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:55.145136 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:56.791228 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:56.791195 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:56.890254 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:56.890215 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:58.773843 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:58.773803 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:58.774262 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:58.773874 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:58.779211 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:58.779185 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:58.917316 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:58.917290 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:57:59.131636 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:59.131555 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:59.230852 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:59.230820 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:57:59.332143 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:57:59.332114 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:58:04.938885 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:04.938830 2531 generic.go:356] "Generic (PLEG): container finished" podID="595851c1-64b8-44ad-b819-f6b635101e2e" containerID="716f76777e30be7eb338b356f7dbb86000c46e6a6491f1e1a94af600b3cf1793" exitCode=0 Jul 24 14:58:04.939264 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:04.938908 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-wjxvg" event={"ID":"595851c1-64b8-44ad-b819-f6b635101e2e","Type":"ContainerDied","Data":"716f76777e30be7eb338b356f7dbb86000c46e6a6491f1e1a94af600b3cf1793"} Jul 24 14:58:04.939264 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:04.939261 2531 scope.go:117] "RemoveContainer" containerID="716f76777e30be7eb338b356f7dbb86000c46e6a6491f1e1a94af600b3cf1793" Jul 24 14:58:05.944922 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:05.944891 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-wjxvg" event={"ID":"595851c1-64b8-44ad-b819-f6b635101e2e","Type":"ContainerStarted","Data":"41dd6c69fcb0f5759bb6623a9d158fb5dd3db4a3be76d9162b9f987eec66e04c"} Jul 24 14:58:07.495183 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.495149 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" podUID="888b99fe-7dd7-499b-9326-37b892a7ef56" containerName="registry" containerID="cri-o://678c1dbbaee2c60d04fa00b88a200e9fedc40654e7afd5e4a648e0e55bf35cb9" gracePeriod=30 Jul 24 14:58:07.745336 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.745282 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:58:07.880916 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.880887 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/888b99fe-7dd7-499b-9326-37b892a7ef56-image-registry-private-configuration\") pod \"888b99fe-7dd7-499b-9326-37b892a7ef56\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " Jul 24 14:58:07.881084 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.880927 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/888b99fe-7dd7-499b-9326-37b892a7ef56-ca-trust-extracted\") pod \"888b99fe-7dd7-499b-9326-37b892a7ef56\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " Jul 24 14:58:07.881084 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.880965 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/888b99fe-7dd7-499b-9326-37b892a7ef56-installation-pull-secrets\") pod \"888b99fe-7dd7-499b-9326-37b892a7ef56\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " Jul 24 14:58:07.881084 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.881015 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-bound-sa-token\") pod \"888b99fe-7dd7-499b-9326-37b892a7ef56\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " Jul 24 14:58:07.881084 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.881052 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-certificates\") pod \"888b99fe-7dd7-499b-9326-37b892a7ef56\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " Jul 24 14:58:07.881084 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.881071 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls\") pod \"888b99fe-7dd7-499b-9326-37b892a7ef56\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " Jul 24 14:58:07.881344 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.881088 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v559v\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-kube-api-access-v559v\") pod \"888b99fe-7dd7-499b-9326-37b892a7ef56\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " Jul 24 14:58:07.881344 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.881108 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/888b99fe-7dd7-499b-9326-37b892a7ef56-trusted-ca\") pod \"888b99fe-7dd7-499b-9326-37b892a7ef56\" (UID: \"888b99fe-7dd7-499b-9326-37b892a7ef56\") " Jul 24 14:58:07.882075 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.881717 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/888b99fe-7dd7-499b-9326-37b892a7ef56-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "888b99fe-7dd7-499b-9326-37b892a7ef56" (UID: "888b99fe-7dd7-499b-9326-37b892a7ef56"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:58:07.882075 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.881735 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "888b99fe-7dd7-499b-9326-37b892a7ef56" (UID: "888b99fe-7dd7-499b-9326-37b892a7ef56"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:58:07.883840 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.883758 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/888b99fe-7dd7-499b-9326-37b892a7ef56-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "888b99fe-7dd7-499b-9326-37b892a7ef56" (UID: "888b99fe-7dd7-499b-9326-37b892a7ef56"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 14:58:07.884372 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.883835 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/888b99fe-7dd7-499b-9326-37b892a7ef56-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "888b99fe-7dd7-499b-9326-37b892a7ef56" (UID: "888b99fe-7dd7-499b-9326-37b892a7ef56"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 14:58:07.884372 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.884036 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "888b99fe-7dd7-499b-9326-37b892a7ef56" (UID: "888b99fe-7dd7-499b-9326-37b892a7ef56"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:58:07.884372 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.884108 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-kube-api-access-v559v" (OuterVolumeSpecName: "kube-api-access-v559v") pod "888b99fe-7dd7-499b-9326-37b892a7ef56" (UID: "888b99fe-7dd7-499b-9326-37b892a7ef56"). InnerVolumeSpecName "kube-api-access-v559v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:58:07.884505 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.884375 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "888b99fe-7dd7-499b-9326-37b892a7ef56" (UID: "888b99fe-7dd7-499b-9326-37b892a7ef56"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:58:07.890202 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.890181 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/888b99fe-7dd7-499b-9326-37b892a7ef56-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "888b99fe-7dd7-499b-9326-37b892a7ef56" (UID: "888b99fe-7dd7-499b-9326-37b892a7ef56"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:58:07.954598 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.954571 2531 generic.go:356] "Generic (PLEG): container finished" podID="888b99fe-7dd7-499b-9326-37b892a7ef56" containerID="678c1dbbaee2c60d04fa00b88a200e9fedc40654e7afd5e4a648e0e55bf35cb9" exitCode=0 Jul 24 14:58:07.954700 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.954616 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" Jul 24 14:58:07.954700 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.954668 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" event={"ID":"888b99fe-7dd7-499b-9326-37b892a7ef56","Type":"ContainerDied","Data":"678c1dbbaee2c60d04fa00b88a200e9fedc40654e7afd5e4a648e0e55bf35cb9"} Jul 24 14:58:07.954816 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.954729 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" event={"ID":"888b99fe-7dd7-499b-9326-37b892a7ef56","Type":"ContainerDied","Data":"f31befdc781502d44878d7d94a9a5d3e30f4f0ccb6b7ba4932a5a94d74266a1a"} Jul 24 14:58:07.954816 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.954759 2531 scope.go:117] "RemoveContainer" containerID="678c1dbbaee2c60d04fa00b88a200e9fedc40654e7afd5e4a648e0e55bf35cb9" Jul 24 14:58:07.963074 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.963059 2531 scope.go:117] "RemoveContainer" containerID="678c1dbbaee2c60d04fa00b88a200e9fedc40654e7afd5e4a648e0e55bf35cb9" Jul 24 14:58:07.963321 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:58:07.963301 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"678c1dbbaee2c60d04fa00b88a200e9fedc40654e7afd5e4a648e0e55bf35cb9\": container with ID starting with 678c1dbbaee2c60d04fa00b88a200e9fedc40654e7afd5e4a648e0e55bf35cb9 not found: ID does not exist" containerID="678c1dbbaee2c60d04fa00b88a200e9fedc40654e7afd5e4a648e0e55bf35cb9" Jul 24 14:58:07.963389 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.963335 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"678c1dbbaee2c60d04fa00b88a200e9fedc40654e7afd5e4a648e0e55bf35cb9"} err="failed to get container status \"678c1dbbaee2c60d04fa00b88a200e9fedc40654e7afd5e4a648e0e55bf35cb9\": rpc error: code = NotFound desc = could not find container \"678c1dbbaee2c60d04fa00b88a200e9fedc40654e7afd5e4a648e0e55bf35cb9\": container with ID starting with 678c1dbbaee2c60d04fa00b88a200e9fedc40654e7afd5e4a648e0e55bf35cb9 not found: ID does not exist" Jul 24 14:58:07.976315 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.976289 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-779758b8b8-dc8j5"] Jul 24 14:58:07.981288 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.981269 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-779758b8b8-dc8j5"] Jul 24 14:58:07.982035 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.982020 2531 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-bound-sa-token\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:58:07.982083 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.982039 2531 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-certificates\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:58:07.982083 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.982049 2531 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-registry-tls\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:58:07.982083 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.982057 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v559v\" (UniqueName: \"kubernetes.io/projected/888b99fe-7dd7-499b-9326-37b892a7ef56-kube-api-access-v559v\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:58:07.982083 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.982066 2531 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/888b99fe-7dd7-499b-9326-37b892a7ef56-trusted-ca\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:58:07.982083 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.982074 2531 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/888b99fe-7dd7-499b-9326-37b892a7ef56-image-registry-private-configuration\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:58:07.982083 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.982083 2531 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/888b99fe-7dd7-499b-9326-37b892a7ef56-ca-trust-extracted\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:58:07.982255 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:07.982091 2531 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/888b99fe-7dd7-499b-9326-37b892a7ef56-installation-pull-secrets\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:58:08.275722 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:08.275687 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="888b99fe-7dd7-499b-9326-37b892a7ef56" path="/var/lib/kubelet/pods/888b99fe-7dd7-499b-9326-37b892a7ef56/volumes" Jul 24 14:58:10.042401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:10.042361 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:58:10.141840 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:10.141810 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:58:12.691204 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:12.691152 2531 patch_prober.go:28] interesting pod/image-registry-779758b8b8-dc8j5 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="Get \"https://10.132.0.8:5000/healthz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 14:58:12.691603 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:12.691233 2531 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-779758b8b8-dc8j5" podUID="888b99fe-7dd7-499b-9326-37b892a7ef56" containerName="registry" probeResult="failure" output="Get \"https://10.132.0.8:5000/healthz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Jul 24 14:58:24.020553 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:24.020520 2531 generic.go:356] "Generic (PLEG): container finished" podID="91fda333-13a3-4954-8c20-e3c8c8bd4da3" containerID="ff5e22fa0eae843794b50eac9d79df076552ecd7619c62b1ee322bc7843bfae4" exitCode=0 Jul 24 14:58:24.020962 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:24.020584 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc" event={"ID":"91fda333-13a3-4954-8c20-e3c8c8bd4da3","Type":"ContainerDied","Data":"ff5e22fa0eae843794b50eac9d79df076552ecd7619c62b1ee322bc7843bfae4"} Jul 24 14:58:24.020962 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:24.020899 2531 scope.go:117] "RemoveContainer" containerID="ff5e22fa0eae843794b50eac9d79df076552ecd7619c62b1ee322bc7843bfae4" Jul 24 14:58:25.026410 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:25.026377 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-pvzhc" event={"ID":"91fda333-13a3-4954-8c20-e3c8c8bd4da3","Type":"ContainerStarted","Data":"e5f57a17411c298516a3410241a11a53e18b4758f96bc184be986a93dd612243"} Jul 24 14:58:29.043454 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:29.043415 2531 generic.go:356] "Generic (PLEG): container finished" podID="079183d0-ba31-4d48-aa8f-e11069836062" containerID="6782ae44238d268fd438f8812fd29dee44052f09e0348a0a50010a5960cfa82f" exitCode=0 Jul 24 14:58:29.043851 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:29.043489 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qkp5x" event={"ID":"079183d0-ba31-4d48-aa8f-e11069836062","Type":"ContainerDied","Data":"6782ae44238d268fd438f8812fd29dee44052f09e0348a0a50010a5960cfa82f"} Jul 24 14:58:29.043851 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:29.043807 2531 scope.go:117] "RemoveContainer" containerID="6782ae44238d268fd438f8812fd29dee44052f09e0348a0a50010a5960cfa82f" Jul 24 14:58:30.049553 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:30.049519 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-qkp5x" event={"ID":"079183d0-ba31-4d48-aa8f-e11069836062","Type":"ContainerStarted","Data":"5910c637bd82220c3125d43446243c4e3003b7ae224ccf60ecef165bed9a5fec"} Jul 24 14:58:53.547327 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:53.547276 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:58:53.636075 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:53.636047 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:58:55.948697 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:55.948658 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b588b9796-8vjfp"] Jul 24 14:58:55.949267 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:55.949181 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="888b99fe-7dd7-499b-9326-37b892a7ef56" containerName="registry" Jul 24 14:58:55.949267 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:55.949203 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="888b99fe-7dd7-499b-9326-37b892a7ef56" containerName="registry" Jul 24 14:58:55.949373 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:55.949298 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="888b99fe-7dd7-499b-9326-37b892a7ef56" containerName="registry" Jul 24 14:58:55.953663 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:55.953641 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:55.965877 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:55.965818 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b588b9796-8vjfp"] Jul 24 14:58:56.008469 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.008302 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:58:56.062166 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.062128 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-console-serving-cert\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.062311 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.062190 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-console-config\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.062311 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.062220 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-oauth-serving-cert\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.062311 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.062243 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-trusted-ca-bundle\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.062311 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.062281 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-service-ca\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.062449 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.062341 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnqmc\" (UniqueName: \"kubernetes.io/projected/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-kube-api-access-hnqmc\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.062449 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.062365 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-console-oauth-config\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.112103 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.112072 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:58:56.162887 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.162840 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-console-config\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.162887 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.162894 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-oauth-serving-cert\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.163141 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.162914 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-trusted-ca-bundle\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.163141 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.162931 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-service-ca\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.163141 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.163073 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnqmc\" (UniqueName: \"kubernetes.io/projected/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-kube-api-access-hnqmc\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.163141 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.163120 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-console-oauth-config\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.163326 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.163229 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-console-serving-cert\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.163742 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.163710 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-service-ca\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.163854 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.163722 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-oauth-serving-cert\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.163854 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.163824 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-console-config\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.164142 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.164122 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-trusted-ca-bundle\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.165727 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.165708 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-console-oauth-config\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.165819 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.165801 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-console-serving-cert\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.171195 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.171174 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnqmc\" (UniqueName: \"kubernetes.io/projected/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-kube-api-access-hnqmc\") pod \"console-5b588b9796-8vjfp\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.210105 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.210045 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:58:56.266065 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.266046 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:58:56.400706 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.400685 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b588b9796-8vjfp"] Jul 24 14:58:56.403154 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:58:56.403129 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f6a3c4f_a702_4e35_9ce4_b0abbd0dce0b.slice/crio-5e1309738de87cd97bee8e9e16625784683c8d91ef7fc5cf0a0db9391654a683 WatchSource:0}: Error finding container 5e1309738de87cd97bee8e9e16625784683c8d91ef7fc5cf0a0db9391654a683: Status 404 returned error can't find the container with id 5e1309738de87cd97bee8e9e16625784683c8d91ef7fc5cf0a0db9391654a683 Jul 24 14:58:56.748379 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.748325 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:58:56.823184 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:56.823155 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:58:57.151490 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:57.151408 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b588b9796-8vjfp" event={"ID":"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b","Type":"ContainerStarted","Data":"7d5dc93bb81e4000ccb957598ac4902ff410bba13c07d4a941531a3e85341eac"} Jul 24 14:58:57.151490 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:57.151454 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b588b9796-8vjfp" event={"ID":"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b","Type":"ContainerStarted","Data":"5e1309738de87cd97bee8e9e16625784683c8d91ef7fc5cf0a0db9391654a683"} Jul 24 14:58:57.172702 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:57.172655 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b588b9796-8vjfp" podStartSLOduration=2.172639436 podStartE2EDuration="2.172639436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:58:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:58:57.170993651 +0000 UTC m=+169.401470360" watchObservedRunningTime="2026-07-24 14:58:57.172639436 +0000 UTC m=+169.403116120" Jul 24 14:58:58.014042 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:58.014003 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 14:58:58.014351 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:58.014326 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="alertmanager" containerID="cri-o://a316ac02ad850ab0dc24f44cecd645e2f6175e66db50fae57d9647597d3f5324" gracePeriod=120 Jul 24 14:58:58.014406 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:58.014385 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="kube-rbac-proxy-metric" containerID="cri-o://ae2fc75777b1c6c263b7e4df23eaa5e893d3e78b77d09ab814165b7a49e37863" gracePeriod=120 Jul 24 14:58:58.014466 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:58.014431 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="kube-rbac-proxy-web" containerID="cri-o://2ffde93f0d416d77ccd16359f31468d5a0e26c272691fd8a45514b84db7f4e08" gracePeriod=120 Jul 24 14:58:58.014524 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:58.014452 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="kube-rbac-proxy" containerID="cri-o://d40db5d184bcaf5632770c378fdb92ba178eada086b798ab7cfaf495fa177f81" gracePeriod=120 Jul 24 14:58:58.014524 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:58.014477 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="prom-label-proxy" containerID="cri-o://e8e3d44a3befc34fd6170e2cb3e31151ad0b2bfe6f18c54f3e801417e36a3b84" gracePeriod=120 Jul 24 14:58:58.014623 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:58.014446 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="config-reloader" containerID="cri-o://38b22758f176b9b3566ced6043a003cabf1c84dc65f4e50fe5c4a439495e5eea" gracePeriod=120 Jul 24 14:58:58.159176 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:58.159151 2531 generic.go:356] "Generic (PLEG): container finished" podID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerID="e8e3d44a3befc34fd6170e2cb3e31151ad0b2bfe6f18c54f3e801417e36a3b84" exitCode=0 Jul 24 14:58:58.159176 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:58.159173 2531 generic.go:356] "Generic (PLEG): container finished" podID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerID="d40db5d184bcaf5632770c378fdb92ba178eada086b798ab7cfaf495fa177f81" exitCode=0 Jul 24 14:58:58.159176 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:58.159179 2531 generic.go:356] "Generic (PLEG): container finished" podID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerID="38b22758f176b9b3566ced6043a003cabf1c84dc65f4e50fe5c4a439495e5eea" exitCode=0 Jul 24 14:58:58.159547 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:58.159185 2531 generic.go:356] "Generic (PLEG): container finished" podID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerID="a316ac02ad850ab0dc24f44cecd645e2f6175e66db50fae57d9647597d3f5324" exitCode=0 Jul 24 14:58:58.159547 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:58.159222 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7976279e-37d6-43f2-9f99-be47eb125d4f","Type":"ContainerDied","Data":"e8e3d44a3befc34fd6170e2cb3e31151ad0b2bfe6f18c54f3e801417e36a3b84"} Jul 24 14:58:58.159547 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:58.159282 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7976279e-37d6-43f2-9f99-be47eb125d4f","Type":"ContainerDied","Data":"d40db5d184bcaf5632770c378fdb92ba178eada086b798ab7cfaf495fa177f81"} Jul 24 14:58:58.159547 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:58.159307 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7976279e-37d6-43f2-9f99-be47eb125d4f","Type":"ContainerDied","Data":"38b22758f176b9b3566ced6043a003cabf1c84dc65f4e50fe5c4a439495e5eea"} Jul 24 14:58:58.159547 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:58.159337 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7976279e-37d6-43f2-9f99-be47eb125d4f","Type":"ContainerDied","Data":"a316ac02ad850ab0dc24f44cecd645e2f6175e66db50fae57d9647597d3f5324"} Jul 24 14:58:59.167477 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.167400 2531 generic.go:356] "Generic (PLEG): container finished" podID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerID="ae2fc75777b1c6c263b7e4df23eaa5e893d3e78b77d09ab814165b7a49e37863" exitCode=0 Jul 24 14:58:59.167477 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.167428 2531 generic.go:356] "Generic (PLEG): container finished" podID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerID="2ffde93f0d416d77ccd16359f31468d5a0e26c272691fd8a45514b84db7f4e08" exitCode=0 Jul 24 14:58:59.167477 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.167464 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7976279e-37d6-43f2-9f99-be47eb125d4f","Type":"ContainerDied","Data":"ae2fc75777b1c6c263b7e4df23eaa5e893d3e78b77d09ab814165b7a49e37863"} Jul 24 14:58:59.167932 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.167530 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7976279e-37d6-43f2-9f99-be47eb125d4f","Type":"ContainerDied","Data":"2ffde93f0d416d77ccd16359f31468d5a0e26c272691fd8a45514b84db7f4e08"} Jul 24 14:58:59.264355 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.264330 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:58:59.391706 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.391637 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-main-tls\") pod \"7976279e-37d6-43f2-9f99-be47eb125d4f\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " Jul 24 14:58:59.391706 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.391672 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7976279e-37d6-43f2-9f99-be47eb125d4f-metrics-client-ca\") pod \"7976279e-37d6-43f2-9f99-be47eb125d4f\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " Jul 24 14:58:59.391930 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.391705 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-kube-rbac-proxy\") pod \"7976279e-37d6-43f2-9f99-be47eb125d4f\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " Jul 24 14:58:59.391930 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.391735 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-kube-rbac-proxy-web\") pod \"7976279e-37d6-43f2-9f99-be47eb125d4f\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " Jul 24 14:58:59.391930 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.391758 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7976279e-37d6-43f2-9f99-be47eb125d4f-alertmanager-main-db\") pod \"7976279e-37d6-43f2-9f99-be47eb125d4f\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " Jul 24 14:58:59.391930 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.391789 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7976279e-37d6-43f2-9f99-be47eb125d4f-config-out\") pod \"7976279e-37d6-43f2-9f99-be47eb125d4f\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " Jul 24 14:58:59.391930 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.391831 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7976279e-37d6-43f2-9f99-be47eb125d4f-alertmanager-trusted-ca-bundle\") pod \"7976279e-37d6-43f2-9f99-be47eb125d4f\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " Jul 24 14:58:59.391930 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.391890 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"7976279e-37d6-43f2-9f99-be47eb125d4f\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " Jul 24 14:58:59.392234 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.391945 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-config-volume\") pod \"7976279e-37d6-43f2-9f99-be47eb125d4f\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " Jul 24 14:58:59.392234 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.392020 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-web-config\") pod \"7976279e-37d6-43f2-9f99-be47eb125d4f\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " Jul 24 14:58:59.392234 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.392053 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-cluster-tls-config\") pod \"7976279e-37d6-43f2-9f99-be47eb125d4f\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " Jul 24 14:58:59.392234 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.392085 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7976279e-37d6-43f2-9f99-be47eb125d4f-tls-assets\") pod \"7976279e-37d6-43f2-9f99-be47eb125d4f\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " Jul 24 14:58:59.392234 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.392108 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7976279e-37d6-43f2-9f99-be47eb125d4f-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "7976279e-37d6-43f2-9f99-be47eb125d4f" (UID: "7976279e-37d6-43f2-9f99-be47eb125d4f"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:58:59.392234 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.392162 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sw5b5\" (UniqueName: \"kubernetes.io/projected/7976279e-37d6-43f2-9f99-be47eb125d4f-kube-api-access-sw5b5\") pod \"7976279e-37d6-43f2-9f99-be47eb125d4f\" (UID: \"7976279e-37d6-43f2-9f99-be47eb125d4f\") " Jul 24 14:58:59.392234 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.392159 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7976279e-37d6-43f2-9f99-be47eb125d4f-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "7976279e-37d6-43f2-9f99-be47eb125d4f" (UID: "7976279e-37d6-43f2-9f99-be47eb125d4f"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:58:59.393023 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.392998 2531 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7976279e-37d6-43f2-9f99-be47eb125d4f-metrics-client-ca\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:58:59.393128 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.393028 2531 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7976279e-37d6-43f2-9f99-be47eb125d4f-alertmanager-main-db\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:58:59.393128 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.393050 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7976279e-37d6-43f2-9f99-be47eb125d4f-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "7976279e-37d6-43f2-9f99-be47eb125d4f" (UID: "7976279e-37d6-43f2-9f99-be47eb125d4f"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:58:59.394917 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.394889 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "7976279e-37d6-43f2-9f99-be47eb125d4f" (UID: "7976279e-37d6-43f2-9f99-be47eb125d4f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 14:58:59.395425 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.395388 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "7976279e-37d6-43f2-9f99-be47eb125d4f" (UID: "7976279e-37d6-43f2-9f99-be47eb125d4f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 14:58:59.395518 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.395417 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "7976279e-37d6-43f2-9f99-be47eb125d4f" (UID: "7976279e-37d6-43f2-9f99-be47eb125d4f"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 14:58:59.395825 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.395796 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7976279e-37d6-43f2-9f99-be47eb125d4f-config-out" (OuterVolumeSpecName: "config-out") pod "7976279e-37d6-43f2-9f99-be47eb125d4f" (UID: "7976279e-37d6-43f2-9f99-be47eb125d4f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:58:59.396082 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.396058 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7976279e-37d6-43f2-9f99-be47eb125d4f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7976279e-37d6-43f2-9f99-be47eb125d4f" (UID: "7976279e-37d6-43f2-9f99-be47eb125d4f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:58:59.396082 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.396072 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7976279e-37d6-43f2-9f99-be47eb125d4f-kube-api-access-sw5b5" (OuterVolumeSpecName: "kube-api-access-sw5b5") pod "7976279e-37d6-43f2-9f99-be47eb125d4f" (UID: "7976279e-37d6-43f2-9f99-be47eb125d4f"). InnerVolumeSpecName "kube-api-access-sw5b5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:58:59.396244 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.396181 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-config-volume" (OuterVolumeSpecName: "config-volume") pod "7976279e-37d6-43f2-9f99-be47eb125d4f" (UID: "7976279e-37d6-43f2-9f99-be47eb125d4f"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 14:58:59.396834 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.396813 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "7976279e-37d6-43f2-9f99-be47eb125d4f" (UID: "7976279e-37d6-43f2-9f99-be47eb125d4f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 14:58:59.399946 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.399797 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "7976279e-37d6-43f2-9f99-be47eb125d4f" (UID: "7976279e-37d6-43f2-9f99-be47eb125d4f"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 14:58:59.406891 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.406849 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-web-config" (OuterVolumeSpecName: "web-config") pod "7976279e-37d6-43f2-9f99-be47eb125d4f" (UID: "7976279e-37d6-43f2-9f99-be47eb125d4f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 14:58:59.494042 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.494006 2531 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-config-volume\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:58:59.494042 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.494040 2531 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-web-config\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:58:59.494042 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.494049 2531 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-cluster-tls-config\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:58:59.494215 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.494058 2531 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7976279e-37d6-43f2-9f99-be47eb125d4f-tls-assets\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:58:59.494215 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.494067 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sw5b5\" (UniqueName: \"kubernetes.io/projected/7976279e-37d6-43f2-9f99-be47eb125d4f-kube-api-access-sw5b5\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:58:59.494215 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.494075 2531 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-main-tls\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:58:59.494215 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.494085 2531 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:58:59.494215 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.494095 2531 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:58:59.494215 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.494103 2531 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7976279e-37d6-43f2-9f99-be47eb125d4f-config-out\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:58:59.494215 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.494111 2531 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7976279e-37d6-43f2-9f99-be47eb125d4f-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:58:59.494215 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:58:59.494119 2531 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7976279e-37d6-43f2-9f99-be47eb125d4f-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:59:00.174247 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.174209 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7976279e-37d6-43f2-9f99-be47eb125d4f","Type":"ContainerDied","Data":"fa4a15cc4db219d56d6820f21b94e3c25fbf058d54df345e9a19d3a10edec12f"} Jul 24 14:59:00.174247 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.174238 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.174646 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.174319 2531 scope.go:117] "RemoveContainer" containerID="e8e3d44a3befc34fd6170e2cb3e31151ad0b2bfe6f18c54f3e801417e36a3b84" Jul 24 14:59:00.182775 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.182755 2531 scope.go:117] "RemoveContainer" containerID="ae2fc75777b1c6c263b7e4df23eaa5e893d3e78b77d09ab814165b7a49e37863" Jul 24 14:59:00.190172 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.190152 2531 scope.go:117] "RemoveContainer" containerID="d40db5d184bcaf5632770c378fdb92ba178eada086b798ab7cfaf495fa177f81" Jul 24 14:59:00.196923 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.196903 2531 scope.go:117] "RemoveContainer" containerID="2ffde93f0d416d77ccd16359f31468d5a0e26c272691fd8a45514b84db7f4e08" Jul 24 14:59:00.203424 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.203406 2531 scope.go:117] "RemoveContainer" containerID="38b22758f176b9b3566ced6043a003cabf1c84dc65f4e50fe5c4a439495e5eea" Jul 24 14:59:00.208969 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.208948 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 14:59:00.212781 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.212752 2531 scope.go:117] "RemoveContainer" containerID="a316ac02ad850ab0dc24f44cecd645e2f6175e66db50fae57d9647597d3f5324" Jul 24 14:59:00.219543 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.219522 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 14:59:00.220595 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.220569 2531 scope.go:117] "RemoveContainer" containerID="3eaf70aa34fd7fd636209979fca53e2b2d5b37bd9d0df1988e7c0101d4617fc8" Jul 24 14:59:00.277535 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.277508 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" path="/var/lib/kubelet/pods/7976279e-37d6-43f2-9f99-be47eb125d4f/volumes" Jul 24 14:59:00.278035 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278020 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 14:59:00.278292 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278281 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="alertmanager" Jul 24 14:59:00.278327 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278294 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="alertmanager" Jul 24 14:59:00.278327 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278303 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="prom-label-proxy" Jul 24 14:59:00.278327 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278309 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="prom-label-proxy" Jul 24 14:59:00.278327 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278322 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="init-config-reloader" Jul 24 14:59:00.278327 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278327 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="init-config-reloader" Jul 24 14:59:00.278475 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278340 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="kube-rbac-proxy" Jul 24 14:59:00.278475 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278345 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="kube-rbac-proxy" Jul 24 14:59:00.278475 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278353 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="config-reloader" Jul 24 14:59:00.278475 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278358 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="config-reloader" Jul 24 14:59:00.278475 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278366 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="kube-rbac-proxy-web" Jul 24 14:59:00.278475 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278371 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="kube-rbac-proxy-web" Jul 24 14:59:00.278475 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278377 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="kube-rbac-proxy-metric" Jul 24 14:59:00.278475 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278381 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="kube-rbac-proxy-metric" Jul 24 14:59:00.278475 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278425 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="kube-rbac-proxy" Jul 24 14:59:00.278475 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278433 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="config-reloader" Jul 24 14:59:00.278475 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278437 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="prom-label-proxy" Jul 24 14:59:00.278475 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278444 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="alertmanager" Jul 24 14:59:00.278475 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278450 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="kube-rbac-proxy-metric" Jul 24 14:59:00.278475 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.278455 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="7976279e-37d6-43f2-9f99-be47eb125d4f" containerName="kube-rbac-proxy-web" Jul 24 14:59:00.283519 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.283505 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.286208 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.286186 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 24 14:59:00.286339 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.286323 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 24 14:59:00.286381 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.286343 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 24 14:59:00.287245 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.287228 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 24 14:59:00.287522 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.287500 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 24 14:59:00.287622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.287503 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 24 14:59:00.287622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.287554 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bhgsl\"" Jul 24 14:59:00.287622 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.287559 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 24 14:59:00.288723 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.288699 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:59:00.289533 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.289512 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 24 14:59:00.292909 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.292893 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 24 14:59:00.303126 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.303099 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 14:59:00.374363 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.374285 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:59:00.405267 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.405243 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68002b52-264d-4260-8bbb-eebafa650c9d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.405359 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.405271 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctxqp\" (UniqueName: \"kubernetes.io/projected/68002b52-264d-4260-8bbb-eebafa650c9d-kube-api-access-ctxqp\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.405359 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.405298 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.405359 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.405322 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.405359 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.405355 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68002b52-264d-4260-8bbb-eebafa650c9d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.405554 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.405369 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68002b52-264d-4260-8bbb-eebafa650c9d-config-out\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.405554 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.405395 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-config-volume\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.405554 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.405409 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.405554 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.405423 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-web-config\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.405554 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.405441 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.405554 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.405533 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/68002b52-264d-4260-8bbb-eebafa650c9d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.405755 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.405558 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68002b52-264d-4260-8bbb-eebafa650c9d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.405755 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.405577 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.506576 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.506528 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/68002b52-264d-4260-8bbb-eebafa650c9d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.506576 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.506577 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68002b52-264d-4260-8bbb-eebafa650c9d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.506792 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.506597 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.506792 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.506627 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68002b52-264d-4260-8bbb-eebafa650c9d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.506792 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.506647 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctxqp\" (UniqueName: \"kubernetes.io/projected/68002b52-264d-4260-8bbb-eebafa650c9d-kube-api-access-ctxqp\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.506792 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.506675 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.506792 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.506703 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.506792 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.506738 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68002b52-264d-4260-8bbb-eebafa650c9d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.506792 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.506759 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68002b52-264d-4260-8bbb-eebafa650c9d-config-out\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.507175 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.506806 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-config-volume\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.507175 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.506827 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.507175 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.506850 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-web-config\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.507175 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.506905 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.507624 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.507550 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/68002b52-264d-4260-8bbb-eebafa650c9d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.507724 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.507667 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68002b52-264d-4260-8bbb-eebafa650c9d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.508358 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.508267 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68002b52-264d-4260-8bbb-eebafa650c9d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.510134 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.510058 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-config-volume\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.510484 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.510461 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68002b52-264d-4260-8bbb-eebafa650c9d-config-out\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.510571 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.510560 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68002b52-264d-4260-8bbb-eebafa650c9d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.510784 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.510759 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.510976 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.510958 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-web-config\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.511039 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.510983 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.511105 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.511081 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.511354 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.511338 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.511943 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.511927 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/68002b52-264d-4260-8bbb-eebafa650c9d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.519604 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.519587 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctxqp\" (UniqueName: \"kubernetes.io/projected/68002b52-264d-4260-8bbb-eebafa650c9d-kube-api-access-ctxqp\") pod \"alertmanager-main-0\" (UID: \"68002b52-264d-4260-8bbb-eebafa650c9d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.593687 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.593667 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 14:59:00.735805 ip-10-0-137-108 kubenswrapper[2531]: W0724 14:59:00.735769 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68002b52_264d_4260_8bbb_eebafa650c9d.slice/crio-0adf328416c7b7085f2ceb8c9d5efff990feb0799fc8ded9ec4afa6fcef5b575 WatchSource:0}: Error finding container 0adf328416c7b7085f2ceb8c9d5efff990feb0799fc8ded9ec4afa6fcef5b575: Status 404 returned error can't find the container with id 0adf328416c7b7085f2ceb8c9d5efff990feb0799fc8ded9ec4afa6fcef5b575 Jul 24 14:59:00.736913 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:00.736886 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 14:59:01.180244 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:01.180209 2531 generic.go:356] "Generic (PLEG): container finished" podID="68002b52-264d-4260-8bbb-eebafa650c9d" containerID="fae0b1e951e2c4a7ffe82ddbe2567960c46d173df7939a2de0313a1d80674210" exitCode=0 Jul 24 14:59:01.180657 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:01.180294 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68002b52-264d-4260-8bbb-eebafa650c9d","Type":"ContainerDied","Data":"fae0b1e951e2c4a7ffe82ddbe2567960c46d173df7939a2de0313a1d80674210"} Jul 24 14:59:01.180657 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:01.180364 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68002b52-264d-4260-8bbb-eebafa650c9d","Type":"ContainerStarted","Data":"0adf328416c7b7085f2ceb8c9d5efff990feb0799fc8ded9ec4afa6fcef5b575"} Jul 24 14:59:02.190748 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:02.190705 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68002b52-264d-4260-8bbb-eebafa650c9d","Type":"ContainerStarted","Data":"9f9a3140b66cbb93ea9feabb6266b6d934c9ed1605c3858df5acb193f80f0d46"} Jul 24 14:59:02.191201 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:02.190797 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68002b52-264d-4260-8bbb-eebafa650c9d","Type":"ContainerStarted","Data":"a05e91d4dfa70a32098c22d45e42182fd936f2a60a11422aed66f1a19a4701d6"} Jul 24 14:59:02.191201 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:02.190840 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68002b52-264d-4260-8bbb-eebafa650c9d","Type":"ContainerStarted","Data":"4dece9b7af8186f9efb3324e48be1b943d0e719895ed7e4ad8114253b1fa4b85"} Jul 24 14:59:02.191201 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:02.190907 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68002b52-264d-4260-8bbb-eebafa650c9d","Type":"ContainerStarted","Data":"ead105138b5d525a45efecbd1f8931ac57424d3d6d6d97bf3aefffc4d7f56aa5"} Jul 24 14:59:02.191201 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:02.190946 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68002b52-264d-4260-8bbb-eebafa650c9d","Type":"ContainerStarted","Data":"fb56ac41fda7422500298a29dfd93f206c7d8cccb2e35751a33720b7ce330840"} Jul 24 14:59:02.191201 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:02.190979 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"68002b52-264d-4260-8bbb-eebafa650c9d","Type":"ContainerStarted","Data":"f2a2a1ba4dcf13dc552c37ab0784a269f9c7770804c1ccae82afff7dc68beec4"} Jul 24 14:59:02.223186 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:02.223139 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.208861188 podStartE2EDuration="2.223123464s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.262276ms" isStatefulPod=false podCreationTimestamp="2026-07-24 14:59:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:59:02.22085219 +0000 UTC m=+174.451328921" watchObservedRunningTime="2026-07-24 14:59:02.223123464 +0000 UTC m=+174.453600148" Jul 24 14:59:05.374191 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:05.374153 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:59:05.476636 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:05.476609 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:59:06.266314 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:06.266282 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:59:06.266496 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:06.266336 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:59:06.276312 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:06.276285 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:59:07.216278 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:07.216253 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 14:59:07.266249 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:07.266217 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86bdb6974-zwqhn"] Jul 24 14:59:07.385401 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:07.385370 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:59:07.485539 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:07.485461 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:59:07.552941 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:07.552904 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:59:10.077336 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:10.077301 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:59:10.199024 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:10.198984 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:59:32.287352 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.287318 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-86bdb6974-zwqhn" podUID="9aec1fce-7090-4a2e-a40a-4d087bf54f83" containerName="console" containerID="cri-o://d415ebb88d30000a8fa859c6819ea5b0cf704ac53d0b36e891e51d855331371c" gracePeriod=15 Jul 24 14:59:32.535774 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.535753 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:59:32.571037 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.570973 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-oauth-serving-cert\") pod \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " Jul 24 14:59:32.571037 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.571023 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9aec1fce-7090-4a2e-a40a-4d087bf54f83-console-serving-cert\") pod \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " Jul 24 14:59:32.571224 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.571041 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-service-ca\") pod \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " Jul 24 14:59:32.571224 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.571082 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-console-config\") pod \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " Jul 24 14:59:32.571224 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.571114 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-trusted-ca-bundle\") pod \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " Jul 24 14:59:32.571224 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.571161 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9aec1fce-7090-4a2e-a40a-4d087bf54f83-console-oauth-config\") pod \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " Jul 24 14:59:32.571224 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.571184 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvhtn\" (UniqueName: \"kubernetes.io/projected/9aec1fce-7090-4a2e-a40a-4d087bf54f83-kube-api-access-rvhtn\") pod \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\" (UID: \"9aec1fce-7090-4a2e-a40a-4d087bf54f83\") " Jul 24 14:59:32.571487 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.571461 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9aec1fce-7090-4a2e-a40a-4d087bf54f83" (UID: "9aec1fce-7090-4a2e-a40a-4d087bf54f83"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:59:32.571616 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.571573 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-console-config" (OuterVolumeSpecName: "console-config") pod "9aec1fce-7090-4a2e-a40a-4d087bf54f83" (UID: "9aec1fce-7090-4a2e-a40a-4d087bf54f83"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:59:32.571707 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.571678 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9aec1fce-7090-4a2e-a40a-4d087bf54f83" (UID: "9aec1fce-7090-4a2e-a40a-4d087bf54f83"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:59:32.571767 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.571712 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-service-ca" (OuterVolumeSpecName: "service-ca") pod "9aec1fce-7090-4a2e-a40a-4d087bf54f83" (UID: "9aec1fce-7090-4a2e-a40a-4d087bf54f83"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:59:32.573547 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.573515 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9aec1fce-7090-4a2e-a40a-4d087bf54f83-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9aec1fce-7090-4a2e-a40a-4d087bf54f83" (UID: "9aec1fce-7090-4a2e-a40a-4d087bf54f83"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 14:59:32.573654 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.573585 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9aec1fce-7090-4a2e-a40a-4d087bf54f83-kube-api-access-rvhtn" (OuterVolumeSpecName: "kube-api-access-rvhtn") pod "9aec1fce-7090-4a2e-a40a-4d087bf54f83" (UID: "9aec1fce-7090-4a2e-a40a-4d087bf54f83"). InnerVolumeSpecName "kube-api-access-rvhtn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:59:32.573712 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.573671 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9aec1fce-7090-4a2e-a40a-4d087bf54f83-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9aec1fce-7090-4a2e-a40a-4d087bf54f83" (UID: "9aec1fce-7090-4a2e-a40a-4d087bf54f83"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 14:59:32.672142 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.672105 2531 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-trusted-ca-bundle\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:59:32.672142 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.672133 2531 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9aec1fce-7090-4a2e-a40a-4d087bf54f83-console-oauth-config\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:59:32.672142 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.672147 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvhtn\" (UniqueName: \"kubernetes.io/projected/9aec1fce-7090-4a2e-a40a-4d087bf54f83-kube-api-access-rvhtn\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:59:32.672352 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.672162 2531 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-oauth-serving-cert\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:59:32.672352 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.672174 2531 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9aec1fce-7090-4a2e-a40a-4d087bf54f83-console-serving-cert\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:59:32.672352 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.672186 2531 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-service-ca\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:59:32.672352 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:32.672197 2531 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9aec1fce-7090-4a2e-a40a-4d087bf54f83-console-config\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 14:59:33.322618 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:33.322582 2531 generic.go:356] "Generic (PLEG): container finished" podID="9aec1fce-7090-4a2e-a40a-4d087bf54f83" containerID="d415ebb88d30000a8fa859c6819ea5b0cf704ac53d0b36e891e51d855331371c" exitCode=0 Jul 24 14:59:33.323084 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:33.322624 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86bdb6974-zwqhn" Jul 24 14:59:33.323084 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:33.322630 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86bdb6974-zwqhn" event={"ID":"9aec1fce-7090-4a2e-a40a-4d087bf54f83","Type":"ContainerDied","Data":"d415ebb88d30000a8fa859c6819ea5b0cf704ac53d0b36e891e51d855331371c"} Jul 24 14:59:33.323084 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:33.322675 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86bdb6974-zwqhn" event={"ID":"9aec1fce-7090-4a2e-a40a-4d087bf54f83","Type":"ContainerDied","Data":"feb5d829562fc7ebb5cbb1271b6d9eb43270e79e3a506b9db822c5a0e6f94376"} Jul 24 14:59:33.323084 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:33.322703 2531 scope.go:117] "RemoveContainer" containerID="d415ebb88d30000a8fa859c6819ea5b0cf704ac53d0b36e891e51d855331371c" Jul 24 14:59:33.331425 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:33.331403 2531 scope.go:117] "RemoveContainer" containerID="d415ebb88d30000a8fa859c6819ea5b0cf704ac53d0b36e891e51d855331371c" Jul 24 14:59:33.331676 ip-10-0-137-108 kubenswrapper[2531]: E0724 14:59:33.331656 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d415ebb88d30000a8fa859c6819ea5b0cf704ac53d0b36e891e51d855331371c\": container with ID starting with d415ebb88d30000a8fa859c6819ea5b0cf704ac53d0b36e891e51d855331371c not found: ID does not exist" containerID="d415ebb88d30000a8fa859c6819ea5b0cf704ac53d0b36e891e51d855331371c" Jul 24 14:59:33.331725 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:33.331685 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d415ebb88d30000a8fa859c6819ea5b0cf704ac53d0b36e891e51d855331371c"} err="failed to get container status \"d415ebb88d30000a8fa859c6819ea5b0cf704ac53d0b36e891e51d855331371c\": rpc error: code = NotFound desc = could not find container \"d415ebb88d30000a8fa859c6819ea5b0cf704ac53d0b36e891e51d855331371c\": container with ID starting with d415ebb88d30000a8fa859c6819ea5b0cf704ac53d0b36e891e51d855331371c not found: ID does not exist" Jul 24 14:59:33.343417 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:33.343395 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86bdb6974-zwqhn"] Jul 24 14:59:33.346810 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:33.346792 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86bdb6974-zwqhn"] Jul 24 14:59:34.275662 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:34.275629 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9aec1fce-7090-4a2e-a40a-4d087bf54f83" path="/var/lib/kubelet/pods/9aec1fce-7090-4a2e-a40a-4d087bf54f83/volumes" Jul 24 14:59:35.777198 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:35.777160 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 14:59:35.878008 ip-10-0-137-108 kubenswrapper[2531]: I0724 14:59:35.877965 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:00:14.757954 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:14.757910 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:00:17.900388 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:17.900356 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:00:17.997475 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:17.997442 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:00:18.934710 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:18.934670 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc"] Jul 24 15:00:18.935204 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:18.935183 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9aec1fce-7090-4a2e-a40a-4d087bf54f83" containerName="console" Jul 24 15:00:18.935280 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:18.935207 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="9aec1fce-7090-4a2e-a40a-4d087bf54f83" containerName="console" Jul 24 15:00:18.935336 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:18.935290 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="9aec1fce-7090-4a2e-a40a-4d087bf54f83" containerName="console" Jul 24 15:00:18.939924 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:18.939903 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc" Jul 24 15:00:18.942417 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:18.942395 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 24 15:00:18.942622 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:18.942601 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 24 15:00:18.943280 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:18.943261 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hpjm4\"" Jul 24 15:00:18.951522 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:18.951499 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc"] Jul 24 15:00:19.035139 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:19.035102 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/757482c1-dbd4-466c-a410-d03c5747b916-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc\" (UID: \"757482c1-dbd4-466c-a410-d03c5747b916\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc" Jul 24 15:00:19.035317 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:19.035156 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/757482c1-dbd4-466c-a410-d03c5747b916-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc\" (UID: \"757482c1-dbd4-466c-a410-d03c5747b916\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc" Jul 24 15:00:19.035317 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:19.035179 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pmd6\" (UniqueName: \"kubernetes.io/projected/757482c1-dbd4-466c-a410-d03c5747b916-kube-api-access-2pmd6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc\" (UID: \"757482c1-dbd4-466c-a410-d03c5747b916\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc" Jul 24 15:00:19.136205 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:19.136171 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/757482c1-dbd4-466c-a410-d03c5747b916-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc\" (UID: \"757482c1-dbd4-466c-a410-d03c5747b916\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc" Jul 24 15:00:19.136205 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:19.136209 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pmd6\" (UniqueName: \"kubernetes.io/projected/757482c1-dbd4-466c-a410-d03c5747b916-kube-api-access-2pmd6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc\" (UID: \"757482c1-dbd4-466c-a410-d03c5747b916\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc" Jul 24 15:00:19.136396 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:19.136283 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/757482c1-dbd4-466c-a410-d03c5747b916-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc\" (UID: \"757482c1-dbd4-466c-a410-d03c5747b916\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc" Jul 24 15:00:19.136626 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:19.136602 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/757482c1-dbd4-466c-a410-d03c5747b916-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc\" (UID: \"757482c1-dbd4-466c-a410-d03c5747b916\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc" Jul 24 15:00:19.136666 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:19.136615 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/757482c1-dbd4-466c-a410-d03c5747b916-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc\" (UID: \"757482c1-dbd4-466c-a410-d03c5747b916\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc" Jul 24 15:00:19.145064 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:19.145040 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pmd6\" (UniqueName: \"kubernetes.io/projected/757482c1-dbd4-466c-a410-d03c5747b916-kube-api-access-2pmd6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc\" (UID: \"757482c1-dbd4-466c-a410-d03c5747b916\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc" Jul 24 15:00:19.249350 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:19.249279 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc" Jul 24 15:00:19.378182 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:19.378153 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc"] Jul 24 15:00:19.380630 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:00:19.380600 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod757482c1_dbd4_466c_a410_d03c5747b916.slice/crio-28f2181dd4fe2b6d849c3a09b1968b2b38f26428b6dcdeb3b16250f91fe3b869 WatchSource:0}: Error finding container 28f2181dd4fe2b6d849c3a09b1968b2b38f26428b6dcdeb3b16250f91fe3b869: Status 404 returned error can't find the container with id 28f2181dd4fe2b6d849c3a09b1968b2b38f26428b6dcdeb3b16250f91fe3b869 Jul 24 15:00:19.502794 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:19.502716 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc" event={"ID":"757482c1-dbd4-466c-a410-d03c5747b916","Type":"ContainerStarted","Data":"28f2181dd4fe2b6d849c3a09b1968b2b38f26428b6dcdeb3b16250f91fe3b869"} Jul 24 15:00:27.537752 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:27.537715 2531 generic.go:356] "Generic (PLEG): container finished" podID="757482c1-dbd4-466c-a410-d03c5747b916" containerID="bf0b412fca1daddf9d10c657faf07c62470a890083e5c7814e1a5cd0e835fd5f" exitCode=0 Jul 24 15:00:27.538136 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:27.537753 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc" event={"ID":"757482c1-dbd4-466c-a410-d03c5747b916","Type":"ContainerDied","Data":"bf0b412fca1daddf9d10c657faf07c62470a890083e5c7814e1a5cd0e835fd5f"} Jul 24 15:00:30.552888 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:30.552835 2531 generic.go:356] "Generic (PLEG): container finished" podID="757482c1-dbd4-466c-a410-d03c5747b916" containerID="e3509d8db3d814c52bde60e82ab5a64a827667eff78aeb0f14e7a2375a16b8cb" exitCode=0 Jul 24 15:00:30.553244 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:30.552902 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc" event={"ID":"757482c1-dbd4-466c-a410-d03c5747b916","Type":"ContainerDied","Data":"e3509d8db3d814c52bde60e82ab5a64a827667eff78aeb0f14e7a2375a16b8cb"} Jul 24 15:00:38.590001 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:38.589919 2531 generic.go:356] "Generic (PLEG): container finished" podID="757482c1-dbd4-466c-a410-d03c5747b916" containerID="3d23aa52ba4bdf03f7fef65f52e9ade2bcff7541da2701cbbe773b6166e0d9b6" exitCode=0 Jul 24 15:00:38.590349 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:38.589993 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc" event={"ID":"757482c1-dbd4-466c-a410-d03c5747b916","Type":"ContainerDied","Data":"3d23aa52ba4bdf03f7fef65f52e9ade2bcff7541da2701cbbe773b6166e0d9b6"} Jul 24 15:00:39.729111 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:39.729086 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc" Jul 24 15:00:39.809181 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:39.809152 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/757482c1-dbd4-466c-a410-d03c5747b916-util\") pod \"757482c1-dbd4-466c-a410-d03c5747b916\" (UID: \"757482c1-dbd4-466c-a410-d03c5747b916\") " Jul 24 15:00:39.809342 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:39.809235 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/757482c1-dbd4-466c-a410-d03c5747b916-bundle\") pod \"757482c1-dbd4-466c-a410-d03c5747b916\" (UID: \"757482c1-dbd4-466c-a410-d03c5747b916\") " Jul 24 15:00:39.809342 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:39.809256 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pmd6\" (UniqueName: \"kubernetes.io/projected/757482c1-dbd4-466c-a410-d03c5747b916-kube-api-access-2pmd6\") pod \"757482c1-dbd4-466c-a410-d03c5747b916\" (UID: \"757482c1-dbd4-466c-a410-d03c5747b916\") " Jul 24 15:00:39.809918 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:39.809888 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/757482c1-dbd4-466c-a410-d03c5747b916-bundle" (OuterVolumeSpecName: "bundle") pod "757482c1-dbd4-466c-a410-d03c5747b916" (UID: "757482c1-dbd4-466c-a410-d03c5747b916"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:00:39.811579 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:39.811555 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/757482c1-dbd4-466c-a410-d03c5747b916-kube-api-access-2pmd6" (OuterVolumeSpecName: "kube-api-access-2pmd6") pod "757482c1-dbd4-466c-a410-d03c5747b916" (UID: "757482c1-dbd4-466c-a410-d03c5747b916"). InnerVolumeSpecName "kube-api-access-2pmd6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:00:39.814129 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:39.814105 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/757482c1-dbd4-466c-a410-d03c5747b916-util" (OuterVolumeSpecName: "util") pod "757482c1-dbd4-466c-a410-d03c5747b916" (UID: "757482c1-dbd4-466c-a410-d03c5747b916"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:00:39.909808 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:39.909745 2531 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/757482c1-dbd4-466c-a410-d03c5747b916-bundle\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:00:39.909808 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:39.909769 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pmd6\" (UniqueName: \"kubernetes.io/projected/757482c1-dbd4-466c-a410-d03c5747b916-kube-api-access-2pmd6\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:00:39.909808 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:39.909779 2531 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/757482c1-dbd4-466c-a410-d03c5747b916-util\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:00:40.599761 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:40.599688 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc" event={"ID":"757482c1-dbd4-466c-a410-d03c5747b916","Type":"ContainerDied","Data":"28f2181dd4fe2b6d849c3a09b1968b2b38f26428b6dcdeb3b16250f91fe3b869"} Jul 24 15:00:40.599761 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:40.599706 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xwchc" Jul 24 15:00:40.599761 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:40.599732 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28f2181dd4fe2b6d849c3a09b1968b2b38f26428b6dcdeb3b16250f91fe3b869" Jul 24 15:00:45.230555 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:45.230521 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:00:45.638623 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:45.638537 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:00:47.428153 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.428121 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82cns"] Jul 24 15:00:47.428538 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.428416 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="757482c1-dbd4-466c-a410-d03c5747b916" containerName="util" Jul 24 15:00:47.428538 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.428428 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="757482c1-dbd4-466c-a410-d03c5747b916" containerName="util" Jul 24 15:00:47.428538 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.428440 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="757482c1-dbd4-466c-a410-d03c5747b916" containerName="pull" Jul 24 15:00:47.428538 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.428446 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="757482c1-dbd4-466c-a410-d03c5747b916" containerName="pull" Jul 24 15:00:47.428538 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.428455 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="757482c1-dbd4-466c-a410-d03c5747b916" containerName="extract" Jul 24 15:00:47.428538 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.428462 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="757482c1-dbd4-466c-a410-d03c5747b916" containerName="extract" Jul 24 15:00:47.428538 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.428521 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="757482c1-dbd4-466c-a410-d03c5747b916" containerName="extract" Jul 24 15:00:47.432225 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.432207 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82cns" Jul 24 15:00:47.434711 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.434682 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 24 15:00:47.434829 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.434750 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 24 15:00:47.434957 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.434941 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-zbblk\"" Jul 24 15:00:47.445400 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.445378 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82cns"] Jul 24 15:00:47.459265 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.459238 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1274b9e-7fcc-4806-9207-8af018a159ab-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-82cns\" (UID: \"e1274b9e-7fcc-4806-9207-8af018a159ab\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82cns" Jul 24 15:00:47.459370 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.459276 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jznm7\" (UniqueName: \"kubernetes.io/projected/e1274b9e-7fcc-4806-9207-8af018a159ab-kube-api-access-jznm7\") pod \"cert-manager-operator-controller-manager-55c444bf86-82cns\" (UID: \"e1274b9e-7fcc-4806-9207-8af018a159ab\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82cns" Jul 24 15:00:47.560546 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.560519 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1274b9e-7fcc-4806-9207-8af018a159ab-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-82cns\" (UID: \"e1274b9e-7fcc-4806-9207-8af018a159ab\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82cns" Jul 24 15:00:47.560698 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.560563 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jznm7\" (UniqueName: \"kubernetes.io/projected/e1274b9e-7fcc-4806-9207-8af018a159ab-kube-api-access-jznm7\") pod \"cert-manager-operator-controller-manager-55c444bf86-82cns\" (UID: \"e1274b9e-7fcc-4806-9207-8af018a159ab\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82cns" Jul 24 15:00:47.561004 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.560985 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1274b9e-7fcc-4806-9207-8af018a159ab-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-82cns\" (UID: \"e1274b9e-7fcc-4806-9207-8af018a159ab\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82cns" Jul 24 15:00:47.571448 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.571425 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jznm7\" (UniqueName: \"kubernetes.io/projected/e1274b9e-7fcc-4806-9207-8af018a159ab-kube-api-access-jznm7\") pod \"cert-manager-operator-controller-manager-55c444bf86-82cns\" (UID: \"e1274b9e-7fcc-4806-9207-8af018a159ab\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82cns" Jul 24 15:00:47.741244 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.741174 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82cns" Jul 24 15:00:47.876772 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:47.876744 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82cns"] Jul 24 15:00:47.880426 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:00:47.880401 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1274b9e_7fcc_4806_9207_8af018a159ab.slice/crio-0d4cca1739828c874aa0c14dc277f8c5eb9a8f291abf1c3d2ee3c7d11cd482cc WatchSource:0}: Error finding container 0d4cca1739828c874aa0c14dc277f8c5eb9a8f291abf1c3d2ee3c7d11cd482cc: Status 404 returned error can't find the container with id 0d4cca1739828c874aa0c14dc277f8c5eb9a8f291abf1c3d2ee3c7d11cd482cc Jul 24 15:00:48.146572 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:48.146483 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:00:48.637982 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:48.637943 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82cns" event={"ID":"e1274b9e-7fcc-4806-9207-8af018a159ab","Type":"ContainerStarted","Data":"0d4cca1739828c874aa0c14dc277f8c5eb9a8f291abf1c3d2ee3c7d11cd482cc"} Jul 24 15:00:49.246439 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:49.246395 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:00:49.357426 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:49.357369 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:00:49.644951 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:49.644859 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82cns" event={"ID":"e1274b9e-7fcc-4806-9207-8af018a159ab","Type":"ContainerStarted","Data":"d4b426c25722628280a24e9cc3527637d7396848b136460fdef710fd143eed39"} Jul 24 15:00:49.677380 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:49.677335 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-82cns" podStartSLOduration=1.121703477 podStartE2EDuration="2.677321367s" totalImagesPullingTime="1.55561789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:00:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:00:49.674755726 +0000 UTC m=+281.905232409" watchObservedRunningTime="2026-07-24 15:00:49.677321367 +0000 UTC m=+281.907798050" Jul 24 15:00:50.142575 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:50.142525 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:00:50.344170 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:50.344143 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:00:51.110687 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.110654 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm"] Jul 24 15:00:51.113401 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.113383 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm" Jul 24 15:00:51.115937 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.115915 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 24 15:00:51.116075 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.116060 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hpjm4\"" Jul 24 15:00:51.116655 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.116642 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 24 15:00:51.125833 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.125803 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm"] Jul 24 15:00:51.191586 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.191558 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gvwv\" (UniqueName: \"kubernetes.io/projected/f5f8abdb-5746-4778-8087-3fb5fdd14ae0-kube-api-access-4gvwv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm\" (UID: \"f5f8abdb-5746-4778-8087-3fb5fdd14ae0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm" Jul 24 15:00:51.191716 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.191605 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5f8abdb-5746-4778-8087-3fb5fdd14ae0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm\" (UID: \"f5f8abdb-5746-4778-8087-3fb5fdd14ae0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm" Jul 24 15:00:51.191716 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.191627 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5f8abdb-5746-4778-8087-3fb5fdd14ae0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm\" (UID: \"f5f8abdb-5746-4778-8087-3fb5fdd14ae0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm" Jul 24 15:00:51.230213 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.230186 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:00:51.292586 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.292558 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gvwv\" (UniqueName: \"kubernetes.io/projected/f5f8abdb-5746-4778-8087-3fb5fdd14ae0-kube-api-access-4gvwv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm\" (UID: \"f5f8abdb-5746-4778-8087-3fb5fdd14ae0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm" Jul 24 15:00:51.292720 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.292605 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5f8abdb-5746-4778-8087-3fb5fdd14ae0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm\" (UID: \"f5f8abdb-5746-4778-8087-3fb5fdd14ae0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm" Jul 24 15:00:51.292720 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.292626 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5f8abdb-5746-4778-8087-3fb5fdd14ae0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm\" (UID: \"f5f8abdb-5746-4778-8087-3fb5fdd14ae0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm" Jul 24 15:00:51.292989 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.292973 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5f8abdb-5746-4778-8087-3fb5fdd14ae0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm\" (UID: \"f5f8abdb-5746-4778-8087-3fb5fdd14ae0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm" Jul 24 15:00:51.293033 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.292998 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5f8abdb-5746-4778-8087-3fb5fdd14ae0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm\" (UID: \"f5f8abdb-5746-4778-8087-3fb5fdd14ae0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm" Jul 24 15:00:51.314486 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.314459 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gvwv\" (UniqueName: \"kubernetes.io/projected/f5f8abdb-5746-4778-8087-3fb5fdd14ae0-kube-api-access-4gvwv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm\" (UID: \"f5f8abdb-5746-4778-8087-3fb5fdd14ae0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm" Jul 24 15:00:51.356154 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.356128 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:00:51.423130 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.423112 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm" Jul 24 15:00:51.517783 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.517751 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7b6zg"] Jul 24 15:00:51.520826 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.520803 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7b6zg" Jul 24 15:00:51.524515 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.524490 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-kvd8l\"" Jul 24 15:00:51.524769 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.524752 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 24 15:00:51.524858 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.524841 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 24 15:00:51.535476 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.535450 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7b6zg"] Jul 24 15:00:51.563653 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.563634 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm"] Jul 24 15:00:51.565825 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:00:51.565796 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5f8abdb_5746_4778_8087_3fb5fdd14ae0.slice/crio-26e0dd9a9431354df0c016adede9de41f6ee13e7ecda8d81e9196f0d05244f82 WatchSource:0}: Error finding container 26e0dd9a9431354df0c016adede9de41f6ee13e7ecda8d81e9196f0d05244f82: Status 404 returned error can't find the container with id 26e0dd9a9431354df0c016adede9de41f6ee13e7ecda8d81e9196f0d05244f82 Jul 24 15:00:51.595252 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.595172 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfpqk\" (UniqueName: \"kubernetes.io/projected/70d912bf-d9f2-4f7f-8553-fbcadc6c7867-kube-api-access-cfpqk\") pod \"cert-manager-webhook-6486ffd997-7b6zg\" (UID: \"70d912bf-d9f2-4f7f-8553-fbcadc6c7867\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7b6zg" Jul 24 15:00:51.595252 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.595221 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/70d912bf-d9f2-4f7f-8553-fbcadc6c7867-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7b6zg\" (UID: \"70d912bf-d9f2-4f7f-8553-fbcadc6c7867\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7b6zg" Jul 24 15:00:51.654401 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.654372 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm" event={"ID":"f5f8abdb-5746-4778-8087-3fb5fdd14ae0","Type":"ContainerStarted","Data":"bfbdb7f90a3b31a7912a424cdb66bb334a8603e09a4e2768101b8a9160739f06"} Jul 24 15:00:51.654508 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.654432 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm" event={"ID":"f5f8abdb-5746-4778-8087-3fb5fdd14ae0","Type":"ContainerStarted","Data":"26e0dd9a9431354df0c016adede9de41f6ee13e7ecda8d81e9196f0d05244f82"} Jul 24 15:00:51.696012 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.695955 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfpqk\" (UniqueName: \"kubernetes.io/projected/70d912bf-d9f2-4f7f-8553-fbcadc6c7867-kube-api-access-cfpqk\") pod \"cert-manager-webhook-6486ffd997-7b6zg\" (UID: \"70d912bf-d9f2-4f7f-8553-fbcadc6c7867\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7b6zg" Jul 24 15:00:51.696012 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.696005 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/70d912bf-d9f2-4f7f-8553-fbcadc6c7867-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7b6zg\" (UID: \"70d912bf-d9f2-4f7f-8553-fbcadc6c7867\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7b6zg" Jul 24 15:00:51.704201 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.704177 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/70d912bf-d9f2-4f7f-8553-fbcadc6c7867-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7b6zg\" (UID: \"70d912bf-d9f2-4f7f-8553-fbcadc6c7867\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7b6zg" Jul 24 15:00:51.704544 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.704526 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfpqk\" (UniqueName: \"kubernetes.io/projected/70d912bf-d9f2-4f7f-8553-fbcadc6c7867-kube-api-access-cfpqk\") pod \"cert-manager-webhook-6486ffd997-7b6zg\" (UID: \"70d912bf-d9f2-4f7f-8553-fbcadc6c7867\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7b6zg" Jul 24 15:00:51.840645 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.840606 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7b6zg" Jul 24 15:00:51.990400 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:51.990374 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7b6zg"] Jul 24 15:00:51.992590 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:00:51.992563 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70d912bf_d9f2_4f7f_8553_fbcadc6c7867.slice/crio-7e789992237773776542fe50b07be5c5416cac4d78bc9fbedbea8dc67ca5cd49 WatchSource:0}: Error finding container 7e789992237773776542fe50b07be5c5416cac4d78bc9fbedbea8dc67ca5cd49: Status 404 returned error can't find the container with id 7e789992237773776542fe50b07be5c5416cac4d78bc9fbedbea8dc67ca5cd49 Jul 24 15:00:52.664884 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:52.664788 2531 generic.go:356] "Generic (PLEG): container finished" podID="f5f8abdb-5746-4778-8087-3fb5fdd14ae0" containerID="bfbdb7f90a3b31a7912a424cdb66bb334a8603e09a4e2768101b8a9160739f06" exitCode=0 Jul 24 15:00:52.664884 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:52.664849 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm" event={"ID":"f5f8abdb-5746-4778-8087-3fb5fdd14ae0","Type":"ContainerDied","Data":"bfbdb7f90a3b31a7912a424cdb66bb334a8603e09a4e2768101b8a9160739f06"} Jul 24 15:00:52.666218 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:52.666192 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7b6zg" event={"ID":"70d912bf-d9f2-4f7f-8553-fbcadc6c7867","Type":"ContainerStarted","Data":"7e789992237773776542fe50b07be5c5416cac4d78bc9fbedbea8dc67ca5cd49"} Jul 24 15:00:53.676656 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:53.676620 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:00:53.837538 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:53.837430 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:00:55.632405 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:55.632375 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:00:55.733842 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:55.733792 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:00:56.687375 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:56.687336 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7b6zg" event={"ID":"70d912bf-d9f2-4f7f-8553-fbcadc6c7867","Type":"ContainerStarted","Data":"dce872a984e70bc0e6848818e8a07084a022a3e0cbc8300d83ebc1b0b336c4e4"} Jul 24 15:00:56.687825 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:56.687388 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7b6zg" Jul 24 15:00:56.689195 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:56.689168 2531 generic.go:356] "Generic (PLEG): container finished" podID="f5f8abdb-5746-4778-8087-3fb5fdd14ae0" containerID="96d462dfd04beb9d8589cb6903133284d25607dda03d913257aeba81ec1d5f28" exitCode=0 Jul 24 15:00:56.689289 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:56.689223 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm" event={"ID":"f5f8abdb-5746-4778-8087-3fb5fdd14ae0","Type":"ContainerDied","Data":"96d462dfd04beb9d8589cb6903133284d25607dda03d913257aeba81ec1d5f28"} Jul 24 15:00:56.706517 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:56.706479 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-7b6zg" podStartSLOduration=2.01721831 podStartE2EDuration="5.706466231s" totalImagesPullingTime="3.689247921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:00:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:00:56.705180921 +0000 UTC m=+288.935657608" watchObservedRunningTime="2026-07-24 15:00:56.706466231 +0000 UTC m=+288.936942914" Jul 24 15:00:57.695987 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:57.695955 2531 generic.go:356] "Generic (PLEG): container finished" podID="f5f8abdb-5746-4778-8087-3fb5fdd14ae0" containerID="8a3a2ba7d931b97a945f0ce45397e7d1a177938a26cf4c0f0cf8f1b8376af412" exitCode=0 Jul 24 15:00:57.696343 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:57.696020 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm" event={"ID":"f5f8abdb-5746-4778-8087-3fb5fdd14ae0","Type":"ContainerDied","Data":"8a3a2ba7d931b97a945f0ce45397e7d1a177938a26cf4c0f0cf8f1b8376af412"} Jul 24 15:00:58.829059 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:58.829037 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm" Jul 24 15:00:58.854684 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:58.854648 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5f8abdb-5746-4778-8087-3fb5fdd14ae0-bundle\") pod \"f5f8abdb-5746-4778-8087-3fb5fdd14ae0\" (UID: \"f5f8abdb-5746-4778-8087-3fb5fdd14ae0\") " Jul 24 15:00:58.854805 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:58.854748 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5f8abdb-5746-4778-8087-3fb5fdd14ae0-util\") pod \"f5f8abdb-5746-4778-8087-3fb5fdd14ae0\" (UID: \"f5f8abdb-5746-4778-8087-3fb5fdd14ae0\") " Jul 24 15:00:58.854805 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:58.854794 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gvwv\" (UniqueName: \"kubernetes.io/projected/f5f8abdb-5746-4778-8087-3fb5fdd14ae0-kube-api-access-4gvwv\") pod \"f5f8abdb-5746-4778-8087-3fb5fdd14ae0\" (UID: \"f5f8abdb-5746-4778-8087-3fb5fdd14ae0\") " Jul 24 15:00:58.855142 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:58.855117 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5f8abdb-5746-4778-8087-3fb5fdd14ae0-bundle" (OuterVolumeSpecName: "bundle") pod "f5f8abdb-5746-4778-8087-3fb5fdd14ae0" (UID: "f5f8abdb-5746-4778-8087-3fb5fdd14ae0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:00:58.857282 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:58.857255 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5f8abdb-5746-4778-8087-3fb5fdd14ae0-kube-api-access-4gvwv" (OuterVolumeSpecName: "kube-api-access-4gvwv") pod "f5f8abdb-5746-4778-8087-3fb5fdd14ae0" (UID: "f5f8abdb-5746-4778-8087-3fb5fdd14ae0"). InnerVolumeSpecName "kube-api-access-4gvwv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:00:58.859539 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:58.859515 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5f8abdb-5746-4778-8087-3fb5fdd14ae0-util" (OuterVolumeSpecName: "util") pod "f5f8abdb-5746-4778-8087-3fb5fdd14ae0" (UID: "f5f8abdb-5746-4778-8087-3fb5fdd14ae0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:00:58.955384 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:58.955318 2531 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5f8abdb-5746-4778-8087-3fb5fdd14ae0-bundle\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:00:58.955384 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:58.955340 2531 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5f8abdb-5746-4778-8087-3fb5fdd14ae0-util\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:00:58.955384 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:58.955350 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gvwv\" (UniqueName: \"kubernetes.io/projected/f5f8abdb-5746-4778-8087-3fb5fdd14ae0-kube-api-access-4gvwv\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:00:59.706555 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:59.706527 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm" Jul 24 15:00:59.706715 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:59.706522 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqb9sm" event={"ID":"f5f8abdb-5746-4778-8087-3fb5fdd14ae0","Type":"ContainerDied","Data":"26e0dd9a9431354df0c016adede9de41f6ee13e7ecda8d81e9196f0d05244f82"} Jul 24 15:00:59.706715 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:00:59.706688 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26e0dd9a9431354df0c016adede9de41f6ee13e7ecda8d81e9196f0d05244f82" Jul 24 15:01:01.230710 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:01.230680 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:02.698005 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:02.697971 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7b6zg" Jul 24 15:01:03.049768 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.049680 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:03.124774 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.124744 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-x8qkp"] Jul 24 15:01:03.125100 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.125087 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5f8abdb-5746-4778-8087-3fb5fdd14ae0" containerName="util" Jul 24 15:01:03.125169 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.125101 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5f8abdb-5746-4778-8087-3fb5fdd14ae0" containerName="util" Jul 24 15:01:03.125169 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.125116 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5f8abdb-5746-4778-8087-3fb5fdd14ae0" containerName="pull" Jul 24 15:01:03.125169 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.125121 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5f8abdb-5746-4778-8087-3fb5fdd14ae0" containerName="pull" Jul 24 15:01:03.125169 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.125137 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5f8abdb-5746-4778-8087-3fb5fdd14ae0" containerName="extract" Jul 24 15:01:03.125169 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.125142 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5f8abdb-5746-4778-8087-3fb5fdd14ae0" containerName="extract" Jul 24 15:01:03.125312 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.125196 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5f8abdb-5746-4778-8087-3fb5fdd14ae0" containerName="extract" Jul 24 15:01:03.127301 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.127286 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-x8qkp" Jul 24 15:01:03.129485 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.129466 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-hrfxz\"" Jul 24 15:01:03.135634 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.135614 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-x8qkp"] Jul 24 15:01:03.138828 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.138801 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:03.191419 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.191386 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ldjv\" (UniqueName: \"kubernetes.io/projected/af5df4bb-381c-49fc-95bf-fe6d06c057b3-kube-api-access-7ldjv\") pod \"cert-manager-55b4d954d5-x8qkp\" (UID: \"af5df4bb-381c-49fc-95bf-fe6d06c057b3\") " pod="cert-manager/cert-manager-55b4d954d5-x8qkp" Jul 24 15:01:03.191547 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.191438 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af5df4bb-381c-49fc-95bf-fe6d06c057b3-bound-sa-token\") pod \"cert-manager-55b4d954d5-x8qkp\" (UID: \"af5df4bb-381c-49fc-95bf-fe6d06c057b3\") " pod="cert-manager/cert-manager-55b4d954d5-x8qkp" Jul 24 15:01:03.292023 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.291994 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ldjv\" (UniqueName: \"kubernetes.io/projected/af5df4bb-381c-49fc-95bf-fe6d06c057b3-kube-api-access-7ldjv\") pod \"cert-manager-55b4d954d5-x8qkp\" (UID: \"af5df4bb-381c-49fc-95bf-fe6d06c057b3\") " pod="cert-manager/cert-manager-55b4d954d5-x8qkp" Jul 24 15:01:03.292148 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.292039 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af5df4bb-381c-49fc-95bf-fe6d06c057b3-bound-sa-token\") pod \"cert-manager-55b4d954d5-x8qkp\" (UID: \"af5df4bb-381c-49fc-95bf-fe6d06c057b3\") " pod="cert-manager/cert-manager-55b4d954d5-x8qkp" Jul 24 15:01:03.300903 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.300830 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/af5df4bb-381c-49fc-95bf-fe6d06c057b3-bound-sa-token\") pod \"cert-manager-55b4d954d5-x8qkp\" (UID: \"af5df4bb-381c-49fc-95bf-fe6d06c057b3\") " pod="cert-manager/cert-manager-55b4d954d5-x8qkp" Jul 24 15:01:03.301003 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.300935 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ldjv\" (UniqueName: \"kubernetes.io/projected/af5df4bb-381c-49fc-95bf-fe6d06c057b3-kube-api-access-7ldjv\") pod \"cert-manager-55b4d954d5-x8qkp\" (UID: \"af5df4bb-381c-49fc-95bf-fe6d06c057b3\") " pod="cert-manager/cert-manager-55b4d954d5-x8qkp" Jul 24 15:01:03.437264 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.437229 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-x8qkp" Jul 24 15:01:03.596197 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.596174 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-x8qkp"] Jul 24 15:01:03.598643 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:01:03.598616 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf5df4bb_381c_49fc_95bf_fe6d06c057b3.slice/crio-d6d19bccf69adabb212cf0262a5a7e9515c8354954bd723cb52e048d88b54f21 WatchSource:0}: Error finding container d6d19bccf69adabb212cf0262a5a7e9515c8354954bd723cb52e048d88b54f21: Status 404 returned error can't find the container with id d6d19bccf69adabb212cf0262a5a7e9515c8354954bd723cb52e048d88b54f21 Jul 24 15:01:03.726574 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.726539 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-x8qkp" event={"ID":"af5df4bb-381c-49fc-95bf-fe6d06c057b3","Type":"ContainerStarted","Data":"a371888a613119f77b8037e949f809a800b38a4afab80341bd3328f4f434d28a"} Jul 24 15:01:03.727013 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.726584 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-x8qkp" event={"ID":"af5df4bb-381c-49fc-95bf-fe6d06c057b3","Type":"ContainerStarted","Data":"d6d19bccf69adabb212cf0262a5a7e9515c8354954bd723cb52e048d88b54f21"} Jul 24 15:01:03.747021 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:03.746965 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-x8qkp" podStartSLOduration=0.746946629 podStartE2EDuration="746.946629ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:01:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:01:03.746479192 +0000 UTC m=+295.976955890" watchObservedRunningTime="2026-07-24 15:01:03.746946629 +0000 UTC m=+295.977423316" Jul 24 15:01:04.530891 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:04.530048 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gdm44"] Jul 24 15:01:04.540789 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:04.540761 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gdm44" Jul 24 15:01:04.543757 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:04.543732 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 24 15:01:04.543915 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:04.543796 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-tb79z\"" Jul 24 15:01:04.544304 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:04.544273 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 24 15:01:04.544491 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:04.544470 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gdm44"] Jul 24 15:01:04.609847 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:04.609823 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j72kq\" (UniqueName: \"kubernetes.io/projected/94b58c1f-e723-4430-8847-0891446cb82d-kube-api-access-j72kq\") pod \"openshift-lws-operator-fd8ccff4c-gdm44\" (UID: \"94b58c1f-e723-4430-8847-0891446cb82d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gdm44" Jul 24 15:01:04.609982 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:04.609909 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94b58c1f-e723-4430-8847-0891446cb82d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gdm44\" (UID: \"94b58c1f-e723-4430-8847-0891446cb82d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gdm44" Jul 24 15:01:04.711127 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:04.711096 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94b58c1f-e723-4430-8847-0891446cb82d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gdm44\" (UID: \"94b58c1f-e723-4430-8847-0891446cb82d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gdm44" Jul 24 15:01:04.711287 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:04.711154 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j72kq\" (UniqueName: \"kubernetes.io/projected/94b58c1f-e723-4430-8847-0891446cb82d-kube-api-access-j72kq\") pod \"openshift-lws-operator-fd8ccff4c-gdm44\" (UID: \"94b58c1f-e723-4430-8847-0891446cb82d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gdm44" Jul 24 15:01:04.711506 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:04.711486 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94b58c1f-e723-4430-8847-0891446cb82d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gdm44\" (UID: \"94b58c1f-e723-4430-8847-0891446cb82d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gdm44" Jul 24 15:01:04.723219 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:04.723198 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j72kq\" (UniqueName: \"kubernetes.io/projected/94b58c1f-e723-4430-8847-0891446cb82d-kube-api-access-j72kq\") pod \"openshift-lws-operator-fd8ccff4c-gdm44\" (UID: \"94b58c1f-e723-4430-8847-0891446cb82d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gdm44" Jul 24 15:01:04.851853 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:04.851793 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gdm44" Jul 24 15:01:05.021499 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:05.021468 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gdm44"] Jul 24 15:01:05.024598 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:01:05.024574 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod94b58c1f_e723_4430_8847_0891446cb82d.slice/crio-f58b000ce96dc6c2ef2fc6e91f4c9372cb3f4ca0a23f8e76c8006d54cc6eef6d WatchSource:0}: Error finding container f58b000ce96dc6c2ef2fc6e91f4c9372cb3f4ca0a23f8e76c8006d54cc6eef6d: Status 404 returned error can't find the container with id f58b000ce96dc6c2ef2fc6e91f4c9372cb3f4ca0a23f8e76c8006d54cc6eef6d Jul 24 15:01:05.737500 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:05.737467 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gdm44" event={"ID":"94b58c1f-e723-4430-8847-0891446cb82d","Type":"ContainerStarted","Data":"f58b000ce96dc6c2ef2fc6e91f4c9372cb3f4ca0a23f8e76c8006d54cc6eef6d"} Jul 24 15:01:07.750837 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:07.750755 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gdm44" event={"ID":"94b58c1f-e723-4430-8847-0891446cb82d","Type":"ContainerStarted","Data":"b5f24ed84d3c57432ed07d16c129c5c9fc44b0b5082bc964df5b13b3d8aab2e4"} Jul 24 15:01:07.770373 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:07.770320 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gdm44" podStartSLOduration=1.370770195 podStartE2EDuration="3.770305997s" totalImagesPullingTime="2.399535802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:01:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:01:07.768891585 +0000 UTC m=+299.999368268" watchObservedRunningTime="2026-07-24 15:01:07.770305997 +0000 UTC m=+300.000782681" Jul 24 15:01:08.154125 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:08.154056 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:01:08.156422 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:08.156396 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:01:08.164452 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:08.164431 2531 kubelet.go:1648] "Image garbage collection succeeded" Jul 24 15:01:08.479431 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:08.479392 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:08.937196 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:08.588328 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:10.126634 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.126597 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj"] Jul 24 15:01:10.150300 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.150276 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj"] Jul 24 15:01:10.150429 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.150370 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj" Jul 24 15:01:10.153253 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.153234 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 24 15:01:10.153360 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.153237 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 24 15:01:10.154068 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.154056 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hpjm4\"" Jul 24 15:01:10.258945 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.258911 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/504f1705-ea42-45fa-a406-f45daced71aa-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj\" (UID: \"504f1705-ea42-45fa-a406-f45daced71aa\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj" Jul 24 15:01:10.259119 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.258960 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/504f1705-ea42-45fa-a406-f45daced71aa-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj\" (UID: \"504f1705-ea42-45fa-a406-f45daced71aa\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj" Jul 24 15:01:10.259119 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.259014 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r249\" (UniqueName: \"kubernetes.io/projected/504f1705-ea42-45fa-a406-f45daced71aa-kube-api-access-2r249\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj\" (UID: \"504f1705-ea42-45fa-a406-f45daced71aa\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj" Jul 24 15:01:10.360029 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.359994 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r249\" (UniqueName: \"kubernetes.io/projected/504f1705-ea42-45fa-a406-f45daced71aa-kube-api-access-2r249\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj\" (UID: \"504f1705-ea42-45fa-a406-f45daced71aa\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj" Jul 24 15:01:10.360196 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.360109 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/504f1705-ea42-45fa-a406-f45daced71aa-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj\" (UID: \"504f1705-ea42-45fa-a406-f45daced71aa\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj" Jul 24 15:01:10.360196 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.360142 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/504f1705-ea42-45fa-a406-f45daced71aa-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj\" (UID: \"504f1705-ea42-45fa-a406-f45daced71aa\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj" Jul 24 15:01:10.360501 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.360484 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/504f1705-ea42-45fa-a406-f45daced71aa-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj\" (UID: \"504f1705-ea42-45fa-a406-f45daced71aa\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj" Jul 24 15:01:10.360552 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.360507 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/504f1705-ea42-45fa-a406-f45daced71aa-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj\" (UID: \"504f1705-ea42-45fa-a406-f45daced71aa\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj" Jul 24 15:01:10.368540 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.368516 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r249\" (UniqueName: \"kubernetes.io/projected/504f1705-ea42-45fa-a406-f45daced71aa-kube-api-access-2r249\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj\" (UID: \"504f1705-ea42-45fa-a406-f45daced71aa\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj" Jul 24 15:01:10.459719 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.459692 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj" Jul 24 15:01:10.598026 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.598003 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj"] Jul 24 15:01:10.599944 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:01:10.599915 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod504f1705_ea42_45fa_a406_f45daced71aa.slice/crio-665ce0c154059a669eefa212f8a995814556ca910a48fc8fc85579ea59459f1d WatchSource:0}: Error finding container 665ce0c154059a669eefa212f8a995814556ca910a48fc8fc85579ea59459f1d: Status 404 returned error can't find the container with id 665ce0c154059a669eefa212f8a995814556ca910a48fc8fc85579ea59459f1d Jul 24 15:01:10.765577 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.765518 2531 generic.go:356] "Generic (PLEG): container finished" podID="504f1705-ea42-45fa-a406-f45daced71aa" containerID="9144223c6da94f20bcfe5bd65c3dd884b39b7b81c9cfb865c9540ca2347f2e4b" exitCode=0 Jul 24 15:01:10.765697 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.765613 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj" event={"ID":"504f1705-ea42-45fa-a406-f45daced71aa","Type":"ContainerDied","Data":"9144223c6da94f20bcfe5bd65c3dd884b39b7b81c9cfb865c9540ca2347f2e4b"} Jul 24 15:01:10.765697 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.765669 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj" event={"ID":"504f1705-ea42-45fa-a406-f45daced71aa","Type":"ContainerStarted","Data":"665ce0c154059a669eefa212f8a995814556ca910a48fc8fc85579ea59459f1d"} Jul 24 15:01:10.766042 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:10.766027 2531 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 15:01:11.196706 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:11.196668 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:11.771914 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:11.771850 2531 generic.go:356] "Generic (PLEG): container finished" podID="504f1705-ea42-45fa-a406-f45daced71aa" containerID="595a1a6e96ab996d8500d0051c7621a7a93c918da055a45d3dab4c8cd896a068" exitCode=0 Jul 24 15:01:11.772076 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:11.771919 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj" event={"ID":"504f1705-ea42-45fa-a406-f45daced71aa","Type":"ContainerDied","Data":"595a1a6e96ab996d8500d0051c7621a7a93c918da055a45d3dab4c8cd896a068"} Jul 24 15:01:12.778127 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:12.778093 2531 generic.go:356] "Generic (PLEG): container finished" podID="504f1705-ea42-45fa-a406-f45daced71aa" containerID="b4572460f3dea1ba4794521fc43e8675822446c78a976007bcfdb75887bcdd8b" exitCode=0 Jul 24 15:01:12.778491 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:12.778174 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj" event={"ID":"504f1705-ea42-45fa-a406-f45daced71aa","Type":"ContainerDied","Data":"b4572460f3dea1ba4794521fc43e8675822446c78a976007bcfdb75887bcdd8b"} Jul 24 15:01:13.918280 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:13.918251 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj" Jul 24 15:01:13.990920 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:13.990888 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/504f1705-ea42-45fa-a406-f45daced71aa-util\") pod \"504f1705-ea42-45fa-a406-f45daced71aa\" (UID: \"504f1705-ea42-45fa-a406-f45daced71aa\") " Jul 24 15:01:13.991068 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:13.990956 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/504f1705-ea42-45fa-a406-f45daced71aa-bundle\") pod \"504f1705-ea42-45fa-a406-f45daced71aa\" (UID: \"504f1705-ea42-45fa-a406-f45daced71aa\") " Jul 24 15:01:13.991068 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:13.990990 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2r249\" (UniqueName: \"kubernetes.io/projected/504f1705-ea42-45fa-a406-f45daced71aa-kube-api-access-2r249\") pod \"504f1705-ea42-45fa-a406-f45daced71aa\" (UID: \"504f1705-ea42-45fa-a406-f45daced71aa\") " Jul 24 15:01:13.991668 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:13.991635 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504f1705-ea42-45fa-a406-f45daced71aa-bundle" (OuterVolumeSpecName: "bundle") pod "504f1705-ea42-45fa-a406-f45daced71aa" (UID: "504f1705-ea42-45fa-a406-f45daced71aa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:01:13.993171 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:13.993139 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/504f1705-ea42-45fa-a406-f45daced71aa-kube-api-access-2r249" (OuterVolumeSpecName: "kube-api-access-2r249") pod "504f1705-ea42-45fa-a406-f45daced71aa" (UID: "504f1705-ea42-45fa-a406-f45daced71aa"). InnerVolumeSpecName "kube-api-access-2r249". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:01:13.996361 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:13.996330 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/504f1705-ea42-45fa-a406-f45daced71aa-util" (OuterVolumeSpecName: "util") pod "504f1705-ea42-45fa-a406-f45daced71aa" (UID: "504f1705-ea42-45fa-a406-f45daced71aa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:01:14.092590 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:14.092503 2531 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/504f1705-ea42-45fa-a406-f45daced71aa-bundle\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:01:14.092590 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:14.092533 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2r249\" (UniqueName: \"kubernetes.io/projected/504f1705-ea42-45fa-a406-f45daced71aa-kube-api-access-2r249\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:01:14.092590 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:14.092543 2531 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/504f1705-ea42-45fa-a406-f45daced71aa-util\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:01:14.790300 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:14.790268 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj" Jul 24 15:01:14.790479 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:14.790266 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afw89cj" event={"ID":"504f1705-ea42-45fa-a406-f45daced71aa","Type":"ContainerDied","Data":"665ce0c154059a669eefa212f8a995814556ca910a48fc8fc85579ea59459f1d"} Jul 24 15:01:14.790479 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:14.790396 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="665ce0c154059a669eefa212f8a995814556ca910a48fc8fc85579ea59459f1d" Jul 24 15:01:22.286682 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:22.286628 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:23.429635 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.429603 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c"] Jul 24 15:01:23.430039 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.429967 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="504f1705-ea42-45fa-a406-f45daced71aa" containerName="pull" Jul 24 15:01:23.430039 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.429979 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="504f1705-ea42-45fa-a406-f45daced71aa" containerName="pull" Jul 24 15:01:23.430039 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.429988 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="504f1705-ea42-45fa-a406-f45daced71aa" containerName="extract" Jul 24 15:01:23.430039 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.429993 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="504f1705-ea42-45fa-a406-f45daced71aa" containerName="extract" Jul 24 15:01:23.430039 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.430018 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="504f1705-ea42-45fa-a406-f45daced71aa" containerName="util" Jul 24 15:01:23.430039 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.430023 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="504f1705-ea42-45fa-a406-f45daced71aa" containerName="util" Jul 24 15:01:23.430244 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.430070 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="504f1705-ea42-45fa-a406-f45daced71aa" containerName="extract" Jul 24 15:01:23.432708 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.432692 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" Jul 24 15:01:23.435303 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.435275 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 24 15:01:23.435431 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.435278 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 24 15:01:23.435431 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.435394 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 24 15:01:23.435763 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.435746 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 24 15:01:23.436829 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.436811 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-lmn65\"" Jul 24 15:01:23.463511 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.463476 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c"] Jul 24 15:01:23.569287 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.569251 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rqj9\" (UniqueName: \"kubernetes.io/projected/077904ac-fa0f-4a6c-b19e-9e9b6da1b68f-kube-api-access-8rqj9\") pod \"opendatahub-operator-controller-manager-7f5584866b-6mp2c\" (UID: \"077904ac-fa0f-4a6c-b19e-9e9b6da1b68f\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" Jul 24 15:01:23.569445 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.569313 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/077904ac-fa0f-4a6c-b19e-9e9b6da1b68f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f5584866b-6mp2c\" (UID: \"077904ac-fa0f-4a6c-b19e-9e9b6da1b68f\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" Jul 24 15:01:23.569445 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.569396 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/077904ac-fa0f-4a6c-b19e-9e9b6da1b68f-manifests\") pod \"opendatahub-operator-controller-manager-7f5584866b-6mp2c\" (UID: \"077904ac-fa0f-4a6c-b19e-9e9b6da1b68f\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" Jul 24 15:01:23.569445 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.569425 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/077904ac-fa0f-4a6c-b19e-9e9b6da1b68f-tmp\") pod \"opendatahub-operator-controller-manager-7f5584866b-6mp2c\" (UID: \"077904ac-fa0f-4a6c-b19e-9e9b6da1b68f\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" Jul 24 15:01:23.569572 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.569445 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/077904ac-fa0f-4a6c-b19e-9e9b6da1b68f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f5584866b-6mp2c\" (UID: \"077904ac-fa0f-4a6c-b19e-9e9b6da1b68f\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" Jul 24 15:01:23.670718 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.670687 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/077904ac-fa0f-4a6c-b19e-9e9b6da1b68f-manifests\") pod \"opendatahub-operator-controller-manager-7f5584866b-6mp2c\" (UID: \"077904ac-fa0f-4a6c-b19e-9e9b6da1b68f\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" Jul 24 15:01:23.670718 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.670718 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/077904ac-fa0f-4a6c-b19e-9e9b6da1b68f-tmp\") pod \"opendatahub-operator-controller-manager-7f5584866b-6mp2c\" (UID: \"077904ac-fa0f-4a6c-b19e-9e9b6da1b68f\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" Jul 24 15:01:23.670978 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.670737 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/077904ac-fa0f-4a6c-b19e-9e9b6da1b68f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f5584866b-6mp2c\" (UID: \"077904ac-fa0f-4a6c-b19e-9e9b6da1b68f\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" Jul 24 15:01:23.670978 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.670775 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rqj9\" (UniqueName: \"kubernetes.io/projected/077904ac-fa0f-4a6c-b19e-9e9b6da1b68f-kube-api-access-8rqj9\") pod \"opendatahub-operator-controller-manager-7f5584866b-6mp2c\" (UID: \"077904ac-fa0f-4a6c-b19e-9e9b6da1b68f\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" Jul 24 15:01:23.670978 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.670817 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/077904ac-fa0f-4a6c-b19e-9e9b6da1b68f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f5584866b-6mp2c\" (UID: \"077904ac-fa0f-4a6c-b19e-9e9b6da1b68f\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" Jul 24 15:01:23.671143 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.671088 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/077904ac-fa0f-4a6c-b19e-9e9b6da1b68f-manifests\") pod \"opendatahub-operator-controller-manager-7f5584866b-6mp2c\" (UID: \"077904ac-fa0f-4a6c-b19e-9e9b6da1b68f\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" Jul 24 15:01:23.673201 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.673180 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/077904ac-fa0f-4a6c-b19e-9e9b6da1b68f-tmp\") pod \"opendatahub-operator-controller-manager-7f5584866b-6mp2c\" (UID: \"077904ac-fa0f-4a6c-b19e-9e9b6da1b68f\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" Jul 24 15:01:23.673401 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.673383 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/077904ac-fa0f-4a6c-b19e-9e9b6da1b68f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f5584866b-6mp2c\" (UID: \"077904ac-fa0f-4a6c-b19e-9e9b6da1b68f\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" Jul 24 15:01:23.673456 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.673441 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/077904ac-fa0f-4a6c-b19e-9e9b6da1b68f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f5584866b-6mp2c\" (UID: \"077904ac-fa0f-4a6c-b19e-9e9b6da1b68f\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" Jul 24 15:01:23.686517 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.686471 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rqj9\" (UniqueName: \"kubernetes.io/projected/077904ac-fa0f-4a6c-b19e-9e9b6da1b68f-kube-api-access-8rqj9\") pod \"opendatahub-operator-controller-manager-7f5584866b-6mp2c\" (UID: \"077904ac-fa0f-4a6c-b19e-9e9b6da1b68f\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" Jul 24 15:01:23.743352 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.743329 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" Jul 24 15:01:23.889853 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:23.889680 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c"] Jul 24 15:01:23.892562 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:01:23.892534 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod077904ac_fa0f_4a6c_b19e_9e9b6da1b68f.slice/crio-948d8fd608e61a7d3cd53c07090e125e2706cae36ed93323f79194ad17ca3ca5 WatchSource:0}: Error finding container 948d8fd608e61a7d3cd53c07090e125e2706cae36ed93323f79194ad17ca3ca5: Status 404 returned error can't find the container with id 948d8fd608e61a7d3cd53c07090e125e2706cae36ed93323f79194ad17ca3ca5 Jul 24 15:01:24.815123 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:24.815088 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:24.850698 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:24.850658 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" event={"ID":"077904ac-fa0f-4a6c-b19e-9e9b6da1b68f","Type":"ContainerStarted","Data":"948d8fd608e61a7d3cd53c07090e125e2706cae36ed93323f79194ad17ca3ca5"} Jul 24 15:01:27.867686 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:27.867650 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" event={"ID":"077904ac-fa0f-4a6c-b19e-9e9b6da1b68f","Type":"ContainerStarted","Data":"743209d912ccd83174004c96c57d78644098107656b221ba25e2b970d9745a0b"} Jul 24 15:01:28.028718 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:28.028691 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:28.130892 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:28.130799 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:28.873856 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:28.873824 2531 generic.go:356] "Generic (PLEG): container finished" podID="077904ac-fa0f-4a6c-b19e-9e9b6da1b68f" containerID="743209d912ccd83174004c96c57d78644098107656b221ba25e2b970d9745a0b" exitCode=0 Jul 24 15:01:28.874228 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:28.873908 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" event={"ID":"077904ac-fa0f-4a6c-b19e-9e9b6da1b68f","Type":"ContainerDied","Data":"743209d912ccd83174004c96c57d78644098107656b221ba25e2b970d9745a0b"} Jul 24 15:01:29.882680 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:29.882646 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" event={"ID":"077904ac-fa0f-4a6c-b19e-9e9b6da1b68f","Type":"ContainerStarted","Data":"f1895cfb9e920a9189ba050def7ee5174fa58674172923f396cde69db0f18a4b"} Jul 24 15:01:29.883070 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:29.882725 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" Jul 24 15:01:29.907288 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:29.907236 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" podStartSLOduration=2.538320273 podStartE2EDuration="6.907220556s" totalImagesPullingTime="3.328183329s" totalInitContainerRuntime="1.040716954s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:01:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:01:29.905677154 +0000 UTC m=+322.136153838" watchObservedRunningTime="2026-07-24 15:01:29.907220556 +0000 UTC m=+322.137697239" Jul 24 15:01:33.516474 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:33.516442 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:33.614319 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:33.614287 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:35.567340 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:35.567298 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:35.645915 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:35.645885 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:38.629563 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:38.629525 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:38.713069 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:38.713042 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:40.890015 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:40.889987 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7f5584866b-6mp2c" Jul 24 15:01:42.652910 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:42.652834 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:42.778220 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:42.778182 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:43.588878 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:43.588819 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:43.725111 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:43.725072 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:43.825517 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:43.825487 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:44.366088 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:44.366055 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zk72b"] Jul 24 15:01:44.375756 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:44.375732 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-zk72b" Jul 24 15:01:44.380420 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:44.380396 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-m44sm\"" Jul 24 15:01:44.380623 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:44.380601 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:44.380713 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:44.380638 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 24 15:01:44.381934 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:44.381912 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zk72b"] Jul 24 15:01:44.444838 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:44.444805 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541aed6a-215c-465a-b8ff-dfcd57c9f107-tls-certs\") pod \"model-serving-api-54d89bd5fc-zk72b\" (UID: \"541aed6a-215c-465a-b8ff-dfcd57c9f107\") " pod="opendatahub/model-serving-api-54d89bd5fc-zk72b" Jul 24 15:01:44.445039 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:44.444932 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg2qn\" (UniqueName: \"kubernetes.io/projected/541aed6a-215c-465a-b8ff-dfcd57c9f107-kube-api-access-hg2qn\") pod \"model-serving-api-54d89bd5fc-zk72b\" (UID: \"541aed6a-215c-465a-b8ff-dfcd57c9f107\") " pod="opendatahub/model-serving-api-54d89bd5fc-zk72b" Jul 24 15:01:44.528278 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:44.528239 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:44.546367 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:44.546342 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg2qn\" (UniqueName: \"kubernetes.io/projected/541aed6a-215c-465a-b8ff-dfcd57c9f107-kube-api-access-hg2qn\") pod \"model-serving-api-54d89bd5fc-zk72b\" (UID: \"541aed6a-215c-465a-b8ff-dfcd57c9f107\") " pod="opendatahub/model-serving-api-54d89bd5fc-zk72b" Jul 24 15:01:44.546462 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:44.546389 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541aed6a-215c-465a-b8ff-dfcd57c9f107-tls-certs\") pod \"model-serving-api-54d89bd5fc-zk72b\" (UID: \"541aed6a-215c-465a-b8ff-dfcd57c9f107\") " pod="opendatahub/model-serving-api-54d89bd5fc-zk72b" Jul 24 15:01:44.548960 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:44.548939 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/541aed6a-215c-465a-b8ff-dfcd57c9f107-tls-certs\") pod \"model-serving-api-54d89bd5fc-zk72b\" (UID: \"541aed6a-215c-465a-b8ff-dfcd57c9f107\") " pod="opendatahub/model-serving-api-54d89bd5fc-zk72b" Jul 24 15:01:44.555397 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:44.555377 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg2qn\" (UniqueName: \"kubernetes.io/projected/541aed6a-215c-465a-b8ff-dfcd57c9f107-kube-api-access-hg2qn\") pod \"model-serving-api-54d89bd5fc-zk72b\" (UID: \"541aed6a-215c-465a-b8ff-dfcd57c9f107\") " pod="opendatahub/model-serving-api-54d89bd5fc-zk72b" Jul 24 15:01:44.687030 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:44.686997 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-zk72b" Jul 24 15:01:44.825421 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:44.825393 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zk72b"] Jul 24 15:01:44.828299 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:01:44.828276 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod541aed6a_215c_465a_b8ff_dfcd57c9f107.slice/crio-bffc8c654e02c833465792383d24eda9b24b3af4edd633dddd31a5706969b7ed WatchSource:0}: Error finding container bffc8c654e02c833465792383d24eda9b24b3af4edd633dddd31a5706969b7ed: Status 404 returned error can't find the container with id bffc8c654e02c833465792383d24eda9b24b3af4edd633dddd31a5706969b7ed Jul 24 15:01:44.929372 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:44.929336 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:44.962774 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:44.962690 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-zk72b" event={"ID":"541aed6a-215c-465a-b8ff-dfcd57c9f107","Type":"ContainerStarted","Data":"bffc8c654e02c833465792383d24eda9b24b3af4edd633dddd31a5706969b7ed"} Jul 24 15:01:44.994736 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:44.994703 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-758f749c7-9xg27"] Jul 24 15:01:44.997742 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:44.997724 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-758f749c7-9xg27" Jul 24 15:01:44.999968 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:44.999942 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 24 15:01:45.000050 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:45.000032 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-df8m9\"" Jul 24 15:01:45.000211 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:45.000191 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 24 15:01:45.015170 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:45.015145 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-758f749c7-9xg27"] Jul 24 15:01:45.051770 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:45.051743 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f32962bc-3bd3-41f4-89fe-245a37e1582f-tls-certs\") pod \"kube-auth-proxy-758f749c7-9xg27\" (UID: \"f32962bc-3bd3-41f4-89fe-245a37e1582f\") " pod="openshift-ingress/kube-auth-proxy-758f749c7-9xg27" Jul 24 15:01:45.051965 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:45.051787 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7b75\" (UniqueName: \"kubernetes.io/projected/f32962bc-3bd3-41f4-89fe-245a37e1582f-kube-api-access-m7b75\") pod \"kube-auth-proxy-758f749c7-9xg27\" (UID: \"f32962bc-3bd3-41f4-89fe-245a37e1582f\") " pod="openshift-ingress/kube-auth-proxy-758f749c7-9xg27" Jul 24 15:01:45.051965 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:45.051922 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f32962bc-3bd3-41f4-89fe-245a37e1582f-tmp\") pod \"kube-auth-proxy-758f749c7-9xg27\" (UID: \"f32962bc-3bd3-41f4-89fe-245a37e1582f\") " pod="openshift-ingress/kube-auth-proxy-758f749c7-9xg27" Jul 24 15:01:45.153411 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:45.153372 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7b75\" (UniqueName: \"kubernetes.io/projected/f32962bc-3bd3-41f4-89fe-245a37e1582f-kube-api-access-m7b75\") pod \"kube-auth-proxy-758f749c7-9xg27\" (UID: \"f32962bc-3bd3-41f4-89fe-245a37e1582f\") " pod="openshift-ingress/kube-auth-proxy-758f749c7-9xg27" Jul 24 15:01:45.153580 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:45.153479 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f32962bc-3bd3-41f4-89fe-245a37e1582f-tmp\") pod \"kube-auth-proxy-758f749c7-9xg27\" (UID: \"f32962bc-3bd3-41f4-89fe-245a37e1582f\") " pod="openshift-ingress/kube-auth-proxy-758f749c7-9xg27" Jul 24 15:01:45.153580 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:45.153534 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f32962bc-3bd3-41f4-89fe-245a37e1582f-tls-certs\") pod \"kube-auth-proxy-758f749c7-9xg27\" (UID: \"f32962bc-3bd3-41f4-89fe-245a37e1582f\") " pod="openshift-ingress/kube-auth-proxy-758f749c7-9xg27" Jul 24 15:01:45.153709 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:01:45.153644 2531 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 24 15:01:45.153766 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:01:45.153729 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f32962bc-3bd3-41f4-89fe-245a37e1582f-tls-certs podName:f32962bc-3bd3-41f4-89fe-245a37e1582f nodeName:}" failed. No retries permitted until 2026-07-24 15:01:45.653705876 +0000 UTC m=+337.884182557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/f32962bc-3bd3-41f4-89fe-245a37e1582f-tls-certs") pod "kube-auth-proxy-758f749c7-9xg27" (UID: "f32962bc-3bd3-41f4-89fe-245a37e1582f") : secret "kube-auth-proxy-tls" not found Jul 24 15:01:45.155903 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:45.155884 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f32962bc-3bd3-41f4-89fe-245a37e1582f-tmp\") pod \"kube-auth-proxy-758f749c7-9xg27\" (UID: \"f32962bc-3bd3-41f4-89fe-245a37e1582f\") " pod="openshift-ingress/kube-auth-proxy-758f749c7-9xg27" Jul 24 15:01:45.164432 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:45.164408 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7b75\" (UniqueName: \"kubernetes.io/projected/f32962bc-3bd3-41f4-89fe-245a37e1582f-kube-api-access-m7b75\") pod \"kube-auth-proxy-758f749c7-9xg27\" (UID: \"f32962bc-3bd3-41f4-89fe-245a37e1582f\") " pod="openshift-ingress/kube-auth-proxy-758f749c7-9xg27" Jul 24 15:01:45.306260 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:45.306167 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:45.657680 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:45.657601 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f32962bc-3bd3-41f4-89fe-245a37e1582f-tls-certs\") pod \"kube-auth-proxy-758f749c7-9xg27\" (UID: \"f32962bc-3bd3-41f4-89fe-245a37e1582f\") " pod="openshift-ingress/kube-auth-proxy-758f749c7-9xg27" Jul 24 15:01:45.660536 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:45.660508 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f32962bc-3bd3-41f4-89fe-245a37e1582f-tls-certs\") pod \"kube-auth-proxy-758f749c7-9xg27\" (UID: \"f32962bc-3bd3-41f4-89fe-245a37e1582f\") " pod="openshift-ingress/kube-auth-proxy-758f749c7-9xg27" Jul 24 15:01:45.908288 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:45.908201 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-758f749c7-9xg27" Jul 24 15:01:46.064898 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.064857 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-758f749c7-9xg27"] Jul 24 15:01:46.371209 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.371173 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:46.379854 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.379826 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf"] Jul 24 15:01:46.382235 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.382218 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.385235 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.385210 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 24 15:01:46.385463 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.385442 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 24 15:01:46.385567 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.385490 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-mgbrv\"" Jul 24 15:01:46.385567 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.385500 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 24 15:01:46.385855 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.385838 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 15:01:46.386615 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:01:46.386594 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf32962bc_3bd3_41f4_89fe_245a37e1582f.slice/crio-b2aba465b26dd2046eb75bc0bc6010c19b5803ffc916f43be57914ab46873953 WatchSource:0}: Error finding container b2aba465b26dd2046eb75bc0bc6010c19b5803ffc916f43be57914ab46873953: Status 404 returned error can't find the container with id b2aba465b26dd2046eb75bc0bc6010c19b5803ffc916f43be57914ab46873953 Jul 24 15:01:46.407127 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.407101 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf"] Jul 24 15:01:46.465749 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.465722 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6bae4669-3e05-4e20-b192-53d59885c60a-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.465846 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.465787 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6bae4669-3e05-4e20-b192-53d59885c60a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.465928 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.465907 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6bae4669-3e05-4e20-b192-53d59885c60a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.465969 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.465952 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bae4669-3e05-4e20-b192-53d59885c60a-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.466014 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.465976 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc6vd\" (UniqueName: \"kubernetes.io/projected/6bae4669-3e05-4e20-b192-53d59885c60a-kube-api-access-vc6vd\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.466074 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.466050 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bae4669-3e05-4e20-b192-53d59885c60a-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.466128 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.466092 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6bae4669-3e05-4e20-b192-53d59885c60a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.481064 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.481037 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:46.567242 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.567209 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bae4669-3e05-4e20-b192-53d59885c60a-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.567242 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.567244 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6bae4669-3e05-4e20-b192-53d59885c60a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.567456 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.567290 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6bae4669-3e05-4e20-b192-53d59885c60a-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.567456 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.567321 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6bae4669-3e05-4e20-b192-53d59885c60a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.567456 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.567350 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6bae4669-3e05-4e20-b192-53d59885c60a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.567632 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.567482 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bae4669-3e05-4e20-b192-53d59885c60a-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.567632 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.567536 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc6vd\" (UniqueName: \"kubernetes.io/projected/6bae4669-3e05-4e20-b192-53d59885c60a-kube-api-access-vc6vd\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.567971 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.567944 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6bae4669-3e05-4e20-b192-53d59885c60a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.569912 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.569887 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6bae4669-3e05-4e20-b192-53d59885c60a-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.569912 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.569896 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6bae4669-3e05-4e20-b192-53d59885c60a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.570152 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.570135 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bae4669-3e05-4e20-b192-53d59885c60a-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.570302 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.570285 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6bae4669-3e05-4e20-b192-53d59885c60a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.581363 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.581338 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc6vd\" (UniqueName: \"kubernetes.io/projected/6bae4669-3e05-4e20-b192-53d59885c60a-kube-api-access-vc6vd\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.582391 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.582371 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bae4669-3e05-4e20-b192-53d59885c60a-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-9nqbf\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.700828 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.700794 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:46.870094 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.870067 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf"] Jul 24 15:01:46.872556 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:01:46.872527 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bae4669_3e05_4e20_b192_53d59885c60a.slice/crio-a4feb37911ecb98b97f03bca0b248666aec41aff53c651843de82cfca6967678 WatchSource:0}: Error finding container a4feb37911ecb98b97f03bca0b248666aec41aff53c651843de82cfca6967678: Status 404 returned error can't find the container with id a4feb37911ecb98b97f03bca0b248666aec41aff53c651843de82cfca6967678 Jul 24 15:01:46.979729 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.979637 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" event={"ID":"6bae4669-3e05-4e20-b192-53d59885c60a","Type":"ContainerStarted","Data":"a4feb37911ecb98b97f03bca0b248666aec41aff53c651843de82cfca6967678"} Jul 24 15:01:46.981494 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.981462 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-zk72b" event={"ID":"541aed6a-215c-465a-b8ff-dfcd57c9f107","Type":"ContainerStarted","Data":"0772ad9b207c557bea52841486890761b7d033ccf0e6ea91cf45b11db4cb3b02"} Jul 24 15:01:46.981625 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.981533 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-zk72b" Jul 24 15:01:46.983075 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:46.983048 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-758f749c7-9xg27" event={"ID":"f32962bc-3bd3-41f4-89fe-245a37e1582f","Type":"ContainerStarted","Data":"b2aba465b26dd2046eb75bc0bc6010c19b5803ffc916f43be57914ab46873953"} Jul 24 15:01:47.023041 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:47.022980 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-zk72b" podStartSLOduration=1.43075046 podStartE2EDuration="3.022960119s" totalImagesPullingTime="1.592209659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:01:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:01:47.021832348 +0000 UTC m=+339.252309051" watchObservedRunningTime="2026-07-24 15:01:47.022960119 +0000 UTC m=+339.253436805" Jul 24 15:01:47.961909 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:47.961851 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:49.633787 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:49.633752 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 24 15:01:49.634097 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:49.633823 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 24 15:01:50.001690 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:50.001654 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" event={"ID":"6bae4669-3e05-4e20-b192-53d59885c60a","Type":"ContainerStarted","Data":"fd359ac429998eb53ff123f1f8dbbf538efa0cb381e9308bfc86af35085c2d4f"} Jul 24 15:01:50.001922 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:50.001720 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:50.003283 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:50.003256 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-758f749c7-9xg27" event={"ID":"f32962bc-3bd3-41f4-89fe-245a37e1582f","Type":"ContainerStarted","Data":"e5e8440ef49fc38ffe5a3bcabfbdef76cda6dc2d15b4fef103dde510f200cf66"} Jul 24 15:01:50.003713 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:50.003692 2531 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-9nqbf container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 15:01:50.003780 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:50.003734 2531 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" podUID="6bae4669-3e05-4e20-b192-53d59885c60a" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 15:01:50.027841 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:50.027801 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" podStartSLOduration=1.269019125 podStartE2EDuration="4.027789227s" totalImagesPullingTime="2.758770102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:01:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:01:50.024739486 +0000 UTC m=+342.255216171" watchObservedRunningTime="2026-07-24 15:01:50.027789227 +0000 UTC m=+342.258265911" Jul 24 15:01:50.049617 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:50.049575 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-758f749c7-9xg27" podStartSLOduration=2.805887412 podStartE2EDuration="6.049562813s" totalImagesPullingTime="3.243675401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:01:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:01:50.047778948 +0000 UTC m=+342.278255655" watchObservedRunningTime="2026-07-24 15:01:50.049562813 +0000 UTC m=+342.280039498" Jul 24 15:01:50.447797 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:50.447741 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:50.543219 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:50.543181 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:51.008974 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:51.008924 2531 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-9nqbf container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 15:01:51.009328 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:51.008992 2531 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" podUID="6bae4669-3e05-4e20-b192-53d59885c60a" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 15:01:51.690610 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:51.690573 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:52.745916 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:52.745860 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:52.818706 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:52.818675 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:54.009837 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:54.009807 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:54.988221 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:54.988189 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:55.125351 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.125311 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:55.127891 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.127842 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-ng4q9"] Jul 24 15:01:55.130684 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.130663 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-cs6dm"] Jul 24 15:01:55.130804 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.130768 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ng4q9" Jul 24 15:01:55.133245 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.133226 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs6dm" Jul 24 15:01:55.134435 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.134418 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 24 15:01:55.134519 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.134450 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-2cqrg\"" Jul 24 15:01:55.139154 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.138755 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 24 15:01:55.139223 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.139150 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-bxf48\"" Jul 24 15:01:55.150011 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.149976 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-cs6dm"] Jul 24 15:01:55.151801 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.151771 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-ng4q9"] Jul 24 15:01:55.230471 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.230443 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:55.256191 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.256123 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6060f538-1807-499a-b224-e07dffbebbe8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cs6dm\" (UID: \"6060f538-1807-499a-b224-e07dffbebbe8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs6dm" Jul 24 15:01:55.256191 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.256162 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z9d4\" (UniqueName: \"kubernetes.io/projected/1d69f3f6-a0cc-45aa-8d2d-cc0b4cf5017b-kube-api-access-5z9d4\") pod \"kserve-controller-manager-d6f4b65b8-ng4q9\" (UID: \"1d69f3f6-a0cc-45aa-8d2d-cc0b4cf5017b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-ng4q9" Jul 24 15:01:55.256367 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.256234 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d69f3f6-a0cc-45aa-8d2d-cc0b4cf5017b-cert\") pod \"kserve-controller-manager-d6f4b65b8-ng4q9\" (UID: \"1d69f3f6-a0cc-45aa-8d2d-cc0b4cf5017b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-ng4q9" Jul 24 15:01:55.256367 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.256312 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r58wt\" (UniqueName: \"kubernetes.io/projected/6060f538-1807-499a-b224-e07dffbebbe8-kube-api-access-r58wt\") pod \"llmisvc-controller-manager-777c6bd7d5-cs6dm\" (UID: \"6060f538-1807-499a-b224-e07dffbebbe8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs6dm" Jul 24 15:01:55.357431 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.357403 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6060f538-1807-499a-b224-e07dffbebbe8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cs6dm\" (UID: \"6060f538-1807-499a-b224-e07dffbebbe8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs6dm" Jul 24 15:01:55.357431 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.357431 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z9d4\" (UniqueName: \"kubernetes.io/projected/1d69f3f6-a0cc-45aa-8d2d-cc0b4cf5017b-kube-api-access-5z9d4\") pod \"kserve-controller-manager-d6f4b65b8-ng4q9\" (UID: \"1d69f3f6-a0cc-45aa-8d2d-cc0b4cf5017b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-ng4q9" Jul 24 15:01:55.357632 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.357463 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d69f3f6-a0cc-45aa-8d2d-cc0b4cf5017b-cert\") pod \"kserve-controller-manager-d6f4b65b8-ng4q9\" (UID: \"1d69f3f6-a0cc-45aa-8d2d-cc0b4cf5017b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-ng4q9" Jul 24 15:01:55.357632 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.357491 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r58wt\" (UniqueName: \"kubernetes.io/projected/6060f538-1807-499a-b224-e07dffbebbe8-kube-api-access-r58wt\") pod \"llmisvc-controller-manager-777c6bd7d5-cs6dm\" (UID: \"6060f538-1807-499a-b224-e07dffbebbe8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs6dm" Jul 24 15:01:55.357632 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:01:55.357550 2531 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 24 15:01:55.357632 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:01:55.357620 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6060f538-1807-499a-b224-e07dffbebbe8-cert podName:6060f538-1807-499a-b224-e07dffbebbe8 nodeName:}" failed. No retries permitted until 2026-07-24 15:01:55.857600782 +0000 UTC m=+348.088077450 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6060f538-1807-499a-b224-e07dffbebbe8-cert") pod "llmisvc-controller-manager-777c6bd7d5-cs6dm" (UID: "6060f538-1807-499a-b224-e07dffbebbe8") : secret "llmisvc-webhook-server-cert" not found Jul 24 15:01:55.360013 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.359990 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d69f3f6-a0cc-45aa-8d2d-cc0b4cf5017b-cert\") pod \"kserve-controller-manager-d6f4b65b8-ng4q9\" (UID: \"1d69f3f6-a0cc-45aa-8d2d-cc0b4cf5017b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-ng4q9" Jul 24 15:01:55.381486 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.381461 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r58wt\" (UniqueName: \"kubernetes.io/projected/6060f538-1807-499a-b224-e07dffbebbe8-kube-api-access-r58wt\") pod \"llmisvc-controller-manager-777c6bd7d5-cs6dm\" (UID: \"6060f538-1807-499a-b224-e07dffbebbe8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs6dm" Jul 24 15:01:55.385279 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.385257 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z9d4\" (UniqueName: \"kubernetes.io/projected/1d69f3f6-a0cc-45aa-8d2d-cc0b4cf5017b-kube-api-access-5z9d4\") pod \"kserve-controller-manager-d6f4b65b8-ng4q9\" (UID: \"1d69f3f6-a0cc-45aa-8d2d-cc0b4cf5017b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-ng4q9" Jul 24 15:01:55.397474 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.397453 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:55.443489 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.443466 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ng4q9" Jul 24 15:01:55.637521 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.637481 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:55.721902 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.721857 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-ng4q9"] Jul 24 15:01:55.725780 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:01:55.725734 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d69f3f6_a0cc_45aa_8d2d_cc0b4cf5017b.slice/crio-ca8b8ccd7297b17799ce569e891e4995a035f8eb26f424b607fd23ec534b5520 WatchSource:0}: Error finding container ca8b8ccd7297b17799ce569e891e4995a035f8eb26f424b607fd23ec534b5520: Status 404 returned error can't find the container with id ca8b8ccd7297b17799ce569e891e4995a035f8eb26f424b607fd23ec534b5520 Jul 24 15:01:55.864641 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.864561 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6060f538-1807-499a-b224-e07dffbebbe8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cs6dm\" (UID: \"6060f538-1807-499a-b224-e07dffbebbe8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs6dm" Jul 24 15:01:55.867135 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.867113 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6060f538-1807-499a-b224-e07dffbebbe8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cs6dm\" (UID: \"6060f538-1807-499a-b224-e07dffbebbe8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs6dm" Jul 24 15:01:55.957498 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:55.957467 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:01:56.036191 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:56.036161 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ng4q9" event={"ID":"1d69f3f6-a0cc-45aa-8d2d-cc0b4cf5017b","Type":"ContainerStarted","Data":"ca8b8ccd7297b17799ce569e891e4995a035f8eb26f424b607fd23ec534b5520"} Jul 24 15:01:56.050327 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:56.050305 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs6dm" Jul 24 15:01:56.186528 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:56.186503 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-cs6dm"] Jul 24 15:01:56.188372 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:01:56.188349 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6060f538_1807_499a_b224_e07dffbebbe8.slice/crio-4a6460cec39590ebd0e762a9bacf2ea9046febdcfaf645540cc9a7cd9b1c5ae4 WatchSource:0}: Error finding container 4a6460cec39590ebd0e762a9bacf2ea9046febdcfaf645540cc9a7cd9b1c5ae4: Status 404 returned error can't find the container with id 4a6460cec39590ebd0e762a9bacf2ea9046febdcfaf645540cc9a7cd9b1c5ae4 Jul 24 15:01:57.044593 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:57.044530 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs6dm" event={"ID":"6060f538-1807-499a-b224-e07dffbebbe8","Type":"ContainerStarted","Data":"4a6460cec39590ebd0e762a9bacf2ea9046febdcfaf645540cc9a7cd9b1c5ae4"} Jul 24 15:01:57.995356 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:57.995314 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-zk72b" Jul 24 15:01:58.074846 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.074263 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf"] Jul 24 15:01:58.074846 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.074487 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" podUID="6bae4669-3e05-4e20-b192-53d59885c60a" containerName="discovery" containerID="cri-o://fd359ac429998eb53ff123f1f8dbbf538efa0cb381e9308bfc86af35085c2d4f" gracePeriod=30 Jul 24 15:01:58.563826 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.563795 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:58.691772 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.691742 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6bae4669-3e05-4e20-b192-53d59885c60a-istio-csr-dns-cert\") pod \"6bae4669-3e05-4e20-b192-53d59885c60a\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " Jul 24 15:01:58.691963 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.691780 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bae4669-3e05-4e20-b192-53d59885c60a-local-certs\") pod \"6bae4669-3e05-4e20-b192-53d59885c60a\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " Jul 24 15:01:58.691963 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.691816 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6bae4669-3e05-4e20-b192-53d59885c60a-istio-csr-ca-configmap\") pod \"6bae4669-3e05-4e20-b192-53d59885c60a\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " Jul 24 15:01:58.691963 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.691843 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6bae4669-3e05-4e20-b192-53d59885c60a-istio-kubeconfig\") pod \"6bae4669-3e05-4e20-b192-53d59885c60a\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " Jul 24 15:01:58.691963 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.691901 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6bae4669-3e05-4e20-b192-53d59885c60a-cacerts\") pod \"6bae4669-3e05-4e20-b192-53d59885c60a\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " Jul 24 15:01:58.691963 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.691932 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bae4669-3e05-4e20-b192-53d59885c60a-istio-token\") pod \"6bae4669-3e05-4e20-b192-53d59885c60a\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " Jul 24 15:01:58.692235 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.692016 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vc6vd\" (UniqueName: \"kubernetes.io/projected/6bae4669-3e05-4e20-b192-53d59885c60a-kube-api-access-vc6vd\") pod \"6bae4669-3e05-4e20-b192-53d59885c60a\" (UID: \"6bae4669-3e05-4e20-b192-53d59885c60a\") " Jul 24 15:01:58.692292 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.692262 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bae4669-3e05-4e20-b192-53d59885c60a-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "6bae4669-3e05-4e20-b192-53d59885c60a" (UID: "6bae4669-3e05-4e20-b192-53d59885c60a"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:01:58.695815 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.695752 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bae4669-3e05-4e20-b192-53d59885c60a-cacerts" (OuterVolumeSpecName: "cacerts") pod "6bae4669-3e05-4e20-b192-53d59885c60a" (UID: "6bae4669-3e05-4e20-b192-53d59885c60a"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:01:58.696184 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.696046 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bae4669-3e05-4e20-b192-53d59885c60a-local-certs" (OuterVolumeSpecName: "local-certs") pod "6bae4669-3e05-4e20-b192-53d59885c60a" (UID: "6bae4669-3e05-4e20-b192-53d59885c60a"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:01:58.696184 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.696051 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bae4669-3e05-4e20-b192-53d59885c60a-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "6bae4669-3e05-4e20-b192-53d59885c60a" (UID: "6bae4669-3e05-4e20-b192-53d59885c60a"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:01:58.696764 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.696739 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bae4669-3e05-4e20-b192-53d59885c60a-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "6bae4669-3e05-4e20-b192-53d59885c60a" (UID: "6bae4669-3e05-4e20-b192-53d59885c60a"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:01:58.697434 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.697413 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bae4669-3e05-4e20-b192-53d59885c60a-istio-token" (OuterVolumeSpecName: "istio-token") pod "6bae4669-3e05-4e20-b192-53d59885c60a" (UID: "6bae4669-3e05-4e20-b192-53d59885c60a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:01:58.698357 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.698332 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bae4669-3e05-4e20-b192-53d59885c60a-kube-api-access-vc6vd" (OuterVolumeSpecName: "kube-api-access-vc6vd") pod "6bae4669-3e05-4e20-b192-53d59885c60a" (UID: "6bae4669-3e05-4e20-b192-53d59885c60a"). InnerVolumeSpecName "kube-api-access-vc6vd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:01:58.793313 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.793270 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vc6vd\" (UniqueName: \"kubernetes.io/projected/6bae4669-3e05-4e20-b192-53d59885c60a-kube-api-access-vc6vd\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:01:58.793313 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.793313 2531 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6bae4669-3e05-4e20-b192-53d59885c60a-istio-csr-dns-cert\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:01:58.793528 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.793331 2531 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bae4669-3e05-4e20-b192-53d59885c60a-local-certs\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:01:58.793528 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.793343 2531 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6bae4669-3e05-4e20-b192-53d59885c60a-istio-csr-ca-configmap\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:01:58.793528 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.793356 2531 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6bae4669-3e05-4e20-b192-53d59885c60a-istio-kubeconfig\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:01:58.793528 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.793366 2531 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6bae4669-3e05-4e20-b192-53d59885c60a-cacerts\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:01:58.793528 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:58.793378 2531 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bae4669-3e05-4e20-b192-53d59885c60a-istio-token\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:01:59.059220 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:59.059142 2531 generic.go:356] "Generic (PLEG): container finished" podID="6bae4669-3e05-4e20-b192-53d59885c60a" containerID="fd359ac429998eb53ff123f1f8dbbf538efa0cb381e9308bfc86af35085c2d4f" exitCode=0 Jul 24 15:01:59.059220 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:59.059207 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" Jul 24 15:01:59.059730 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:59.059237 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" event={"ID":"6bae4669-3e05-4e20-b192-53d59885c60a","Type":"ContainerDied","Data":"fd359ac429998eb53ff123f1f8dbbf538efa0cb381e9308bfc86af35085c2d4f"} Jul 24 15:01:59.059730 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:59.059294 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf" event={"ID":"6bae4669-3e05-4e20-b192-53d59885c60a","Type":"ContainerDied","Data":"a4feb37911ecb98b97f03bca0b248666aec41aff53c651843de82cfca6967678"} Jul 24 15:01:59.059730 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:59.059318 2531 scope.go:117] "RemoveContainer" containerID="fd359ac429998eb53ff123f1f8dbbf538efa0cb381e9308bfc86af35085c2d4f" Jul 24 15:01:59.061175 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:59.061150 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ng4q9" event={"ID":"1d69f3f6-a0cc-45aa-8d2d-cc0b4cf5017b","Type":"ContainerStarted","Data":"1144e168a5ebeb5c71ef3475bb845d44e1d56a9bb5e42a2b9a87a766501698c6"} Jul 24 15:01:59.061313 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:59.061294 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ng4q9" Jul 24 15:01:59.094251 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:59.094205 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ng4q9" podStartSLOduration=1.379434581 podStartE2EDuration="4.094186334s" totalImagesPullingTime="2.714751753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:01:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:01:59.092416441 +0000 UTC m=+351.322893124" watchObservedRunningTime="2026-07-24 15:01:59.094186334 +0000 UTC m=+351.324663048" Jul 24 15:01:59.121178 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:59.121148 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf"] Jul 24 15:01:59.126794 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:59.126769 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9nqbf"] Jul 24 15:01:59.277107 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:59.276827 2531 scope.go:117] "RemoveContainer" containerID="fd359ac429998eb53ff123f1f8dbbf538efa0cb381e9308bfc86af35085c2d4f" Jul 24 15:01:59.277205 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:01:59.277160 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd359ac429998eb53ff123f1f8dbbf538efa0cb381e9308bfc86af35085c2d4f\": container with ID starting with fd359ac429998eb53ff123f1f8dbbf538efa0cb381e9308bfc86af35085c2d4f not found: ID does not exist" containerID="fd359ac429998eb53ff123f1f8dbbf538efa0cb381e9308bfc86af35085c2d4f" Jul 24 15:01:59.277205 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:01:59.277193 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd359ac429998eb53ff123f1f8dbbf538efa0cb381e9308bfc86af35085c2d4f"} err="failed to get container status \"fd359ac429998eb53ff123f1f8dbbf538efa0cb381e9308bfc86af35085c2d4f\": rpc error: code = NotFound desc = could not find container \"fd359ac429998eb53ff123f1f8dbbf538efa0cb381e9308bfc86af35085c2d4f\": container with ID starting with fd359ac429998eb53ff123f1f8dbbf538efa0cb381e9308bfc86af35085c2d4f not found: ID does not exist" Jul 24 15:02:00.067405 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:00.067364 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs6dm" event={"ID":"6060f538-1807-499a-b224-e07dffbebbe8","Type":"ContainerStarted","Data":"ed37366da6684cc961fc8917747cc90989b35c243b12a1f481c3f885fec6e208"} Jul 24 15:02:00.067891 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:00.067542 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs6dm" Jul 24 15:02:00.086033 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:00.085984 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs6dm" podStartSLOduration=1.969487494 podStartE2EDuration="5.085973123s" totalImagesPullingTime="3.116485629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:01:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:02:00.084790829 +0000 UTC m=+352.315267513" watchObservedRunningTime="2026-07-24 15:02:00.085973123 +0000 UTC m=+352.316449807" Jul 24 15:02:00.277958 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:00.277921 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bae4669-3e05-4e20-b192-53d59885c60a" path="/var/lib/kubelet/pods/6bae4669-3e05-4e20-b192-53d59885c60a/volumes" Jul 24 15:02:01.184155 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:01.184119 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:02:01.267080 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:01.267040 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:02:30.073582 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:30.073556 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-ng4q9" Jul 24 15:02:31.075828 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:31.075799 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs6dm" Jul 24 15:02:39.971322 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:39.971274 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:02:40.070540 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:40.070506 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:02:41.722252 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:41.722220 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:02:41.822938 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:41.822908 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:02:44.236014 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.235965 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:02:44.333143 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.333116 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:02:44.357255 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.357226 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hzmqv"] Jul 24 15:02:44.357661 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.357644 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bae4669-3e05-4e20-b192-53d59885c60a" containerName="discovery" Jul 24 15:02:44.357750 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.357664 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bae4669-3e05-4e20-b192-53d59885c60a" containerName="discovery" Jul 24 15:02:44.357817 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.357775 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bae4669-3e05-4e20-b192-53d59885c60a" containerName="discovery" Jul 24 15:02:44.361517 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.361494 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hzmqv" Jul 24 15:02:44.364739 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.364717 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 24 15:02:44.365540 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.365520 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-mjwv9\"" Jul 24 15:02:44.365654 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.365545 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 24 15:02:44.373447 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.373420 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hzmqv"] Jul 24 15:02:44.436221 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.436189 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:02:44.474570 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.474537 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2wb2\" (UniqueName: \"kubernetes.io/projected/e5bc025f-a801-4038-8800-54bb1d21a894-kube-api-access-f2wb2\") pod \"kuadrant-operator-catalog-hzmqv\" (UID: \"e5bc025f-a801-4038-8800-54bb1d21a894\") " pod="kuadrant-system/kuadrant-operator-catalog-hzmqv" Jul 24 15:02:44.575242 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.575162 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2wb2\" (UniqueName: \"kubernetes.io/projected/e5bc025f-a801-4038-8800-54bb1d21a894-kube-api-access-f2wb2\") pod \"kuadrant-operator-catalog-hzmqv\" (UID: \"e5bc025f-a801-4038-8800-54bb1d21a894\") " pod="kuadrant-system/kuadrant-operator-catalog-hzmqv" Jul 24 15:02:44.584124 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.584096 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2wb2\" (UniqueName: \"kubernetes.io/projected/e5bc025f-a801-4038-8800-54bb1d21a894-kube-api-access-f2wb2\") pod \"kuadrant-operator-catalog-hzmqv\" (UID: \"e5bc025f-a801-4038-8800-54bb1d21a894\") " pod="kuadrant-system/kuadrant-operator-catalog-hzmqv" Jul 24 15:02:44.673125 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.673087 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hzmqv" Jul 24 15:02:44.706287 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.706255 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hzmqv"] Jul 24 15:02:44.819363 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.819339 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hzmqv"] Jul 24 15:02:44.821004 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:02:44.820977 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5bc025f_a801_4038_8800_54bb1d21a894.slice/crio-ba4fc5fd9e4ac51cd30bab7053bdeffd6ac341628707d2547dd7e1b3c0b15405 WatchSource:0}: Error finding container ba4fc5fd9e4ac51cd30bab7053bdeffd6ac341628707d2547dd7e1b3c0b15405: Status 404 returned error can't find the container with id ba4fc5fd9e4ac51cd30bab7053bdeffd6ac341628707d2547dd7e1b3c0b15405 Jul 24 15:02:44.915894 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.915794 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xlk25"] Jul 24 15:02:44.919021 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.919006 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xlk25" Jul 24 15:02:44.927541 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.927519 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xlk25"] Jul 24 15:02:44.979680 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:44.979647 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8zkq\" (UniqueName: \"kubernetes.io/projected/701e2e43-0ad1-451c-8491-ebf9b2ca9648-kube-api-access-t8zkq\") pod \"kuadrant-operator-catalog-xlk25\" (UID: \"701e2e43-0ad1-451c-8491-ebf9b2ca9648\") " pod="kuadrant-system/kuadrant-operator-catalog-xlk25" Jul 24 15:02:45.080811 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:45.080777 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8zkq\" (UniqueName: \"kubernetes.io/projected/701e2e43-0ad1-451c-8491-ebf9b2ca9648-kube-api-access-t8zkq\") pod \"kuadrant-operator-catalog-xlk25\" (UID: \"701e2e43-0ad1-451c-8491-ebf9b2ca9648\") " pod="kuadrant-system/kuadrant-operator-catalog-xlk25" Jul 24 15:02:45.091277 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:45.091247 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8zkq\" (UniqueName: \"kubernetes.io/projected/701e2e43-0ad1-451c-8491-ebf9b2ca9648-kube-api-access-t8zkq\") pod \"kuadrant-operator-catalog-xlk25\" (UID: \"701e2e43-0ad1-451c-8491-ebf9b2ca9648\") " pod="kuadrant-system/kuadrant-operator-catalog-xlk25" Jul 24 15:02:45.229374 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:45.229344 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xlk25" Jul 24 15:02:45.298890 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:45.298839 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hzmqv" event={"ID":"e5bc025f-a801-4038-8800-54bb1d21a894","Type":"ContainerStarted","Data":"ba4fc5fd9e4ac51cd30bab7053bdeffd6ac341628707d2547dd7e1b3c0b15405"} Jul 24 15:02:45.376270 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:45.376239 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xlk25"] Jul 24 15:02:45.381214 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:02:45.381186 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod701e2e43_0ad1_451c_8491_ebf9b2ca9648.slice/crio-262354250ef4fd4744f2e3db56276c6dbea76c411b2bbfd2af022d06d1321e00 WatchSource:0}: Error finding container 262354250ef4fd4744f2e3db56276c6dbea76c411b2bbfd2af022d06d1321e00: Status 404 returned error can't find the container with id 262354250ef4fd4744f2e3db56276c6dbea76c411b2bbfd2af022d06d1321e00 Jul 24 15:02:46.306828 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:46.306791 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xlk25" event={"ID":"701e2e43-0ad1-451c-8491-ebf9b2ca9648","Type":"ContainerStarted","Data":"262354250ef4fd4744f2e3db56276c6dbea76c411b2bbfd2af022d06d1321e00"} Jul 24 15:02:47.313670 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:47.313628 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xlk25" event={"ID":"701e2e43-0ad1-451c-8491-ebf9b2ca9648","Type":"ContainerStarted","Data":"867f1bcb2d07591a3aa6774f3e55863cbae83f0c3fc5daa6c9a3dda5f1020db4"} Jul 24 15:02:47.315103 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:47.315068 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hzmqv" event={"ID":"e5bc025f-a801-4038-8800-54bb1d21a894","Type":"ContainerStarted","Data":"df4e87428b26d8592ad5ad68fbc7c4d9f66a5a6558ff70910f68bf238064c97d"} Jul 24 15:02:47.315244 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:47.315138 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-hzmqv" podUID="e5bc025f-a801-4038-8800-54bb1d21a894" containerName="registry-server" containerID="cri-o://df4e87428b26d8592ad5ad68fbc7c4d9f66a5a6558ff70910f68bf238064c97d" gracePeriod=2 Jul 24 15:02:47.331541 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:47.331501 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-xlk25" podStartSLOduration=1.837103744 podStartE2EDuration="3.33148936s" totalImagesPullingTime="1.494385616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:02:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:02:47.329681093 +0000 UTC m=+399.560157777" watchObservedRunningTime="2026-07-24 15:02:47.33148936 +0000 UTC m=+399.561966072" Jul 24 15:02:47.346817 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:47.346776 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-hzmqv" podStartSLOduration=1.294892334 podStartE2EDuration="3.346763068s" totalImagesPullingTime="2.051870734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:02:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:02:47.345224868 +0000 UTC m=+399.575701552" watchObservedRunningTime="2026-07-24 15:02:47.346763068 +0000 UTC m=+399.577239752" Jul 24 15:02:47.558289 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:47.558262 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hzmqv" Jul 24 15:02:47.705477 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:47.705449 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2wb2\" (UniqueName: \"kubernetes.io/projected/e5bc025f-a801-4038-8800-54bb1d21a894-kube-api-access-f2wb2\") pod \"e5bc025f-a801-4038-8800-54bb1d21a894\" (UID: \"e5bc025f-a801-4038-8800-54bb1d21a894\") " Jul 24 15:02:47.707665 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:47.707635 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5bc025f-a801-4038-8800-54bb1d21a894-kube-api-access-f2wb2" (OuterVolumeSpecName: "kube-api-access-f2wb2") pod "e5bc025f-a801-4038-8800-54bb1d21a894" (UID: "e5bc025f-a801-4038-8800-54bb1d21a894"). InnerVolumeSpecName "kube-api-access-f2wb2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:02:47.806141 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:47.806116 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2wb2\" (UniqueName: \"kubernetes.io/projected/e5bc025f-a801-4038-8800-54bb1d21a894-kube-api-access-f2wb2\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:02:48.320821 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:48.320791 2531 generic.go:356] "Generic (PLEG): container finished" podID="e5bc025f-a801-4038-8800-54bb1d21a894" containerID="df4e87428b26d8592ad5ad68fbc7c4d9f66a5a6558ff70910f68bf238064c97d" exitCode=0 Jul 24 15:02:48.321245 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:48.320829 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hzmqv" Jul 24 15:02:48.321245 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:48.320883 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hzmqv" event={"ID":"e5bc025f-a801-4038-8800-54bb1d21a894","Type":"ContainerDied","Data":"df4e87428b26d8592ad5ad68fbc7c4d9f66a5a6558ff70910f68bf238064c97d"} Jul 24 15:02:48.321245 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:48.320929 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hzmqv" event={"ID":"e5bc025f-a801-4038-8800-54bb1d21a894","Type":"ContainerDied","Data":"ba4fc5fd9e4ac51cd30bab7053bdeffd6ac341628707d2547dd7e1b3c0b15405"} Jul 24 15:02:48.321245 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:48.320951 2531 scope.go:117] "RemoveContainer" containerID="df4e87428b26d8592ad5ad68fbc7c4d9f66a5a6558ff70910f68bf238064c97d" Jul 24 15:02:48.330056 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:48.330038 2531 scope.go:117] "RemoveContainer" containerID="df4e87428b26d8592ad5ad68fbc7c4d9f66a5a6558ff70910f68bf238064c97d" Jul 24 15:02:48.330304 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:02:48.330287 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df4e87428b26d8592ad5ad68fbc7c4d9f66a5a6558ff70910f68bf238064c97d\": container with ID starting with df4e87428b26d8592ad5ad68fbc7c4d9f66a5a6558ff70910f68bf238064c97d not found: ID does not exist" containerID="df4e87428b26d8592ad5ad68fbc7c4d9f66a5a6558ff70910f68bf238064c97d" Jul 24 15:02:48.330348 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:48.330313 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df4e87428b26d8592ad5ad68fbc7c4d9f66a5a6558ff70910f68bf238064c97d"} err="failed to get container status \"df4e87428b26d8592ad5ad68fbc7c4d9f66a5a6558ff70910f68bf238064c97d\": rpc error: code = NotFound desc = could not find container \"df4e87428b26d8592ad5ad68fbc7c4d9f66a5a6558ff70910f68bf238064c97d\": container with ID starting with df4e87428b26d8592ad5ad68fbc7c4d9f66a5a6558ff70910f68bf238064c97d not found: ID does not exist" Jul 24 15:02:48.337750 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:48.337730 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hzmqv"] Jul 24 15:02:48.341962 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:48.341936 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hzmqv"] Jul 24 15:02:50.276758 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:50.276723 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5bc025f-a801-4038-8800-54bb1d21a894" path="/var/lib/kubelet/pods/e5bc025f-a801-4038-8800-54bb1d21a894/volumes" Jul 24 15:02:52.089903 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.089854 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-dd46c778-5xtfc"] Jul 24 15:02:52.090542 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.090496 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5bc025f-a801-4038-8800-54bb1d21a894" containerName="registry-server" Jul 24 15:02:52.090542 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.090532 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5bc025f-a801-4038-8800-54bb1d21a894" containerName="registry-server" Jul 24 15:02:52.090742 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.090659 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5bc025f-a801-4038-8800-54bb1d21a894" containerName="registry-server" Jul 24 15:02:52.094048 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.094020 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.104489 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.104466 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dd46c778-5xtfc"] Jul 24 15:02:52.156606 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.156582 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:02:52.238214 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.238182 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:02:52.242200 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.242179 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-service-ca\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.242321 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.242221 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-console-serving-cert\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.242321 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.242247 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-console-config\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.242321 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.242274 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-trusted-ca-bundle\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.242455 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.242332 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-console-oauth-config\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.242455 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.242357 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5xq5\" (UniqueName: \"kubernetes.io/projected/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-kube-api-access-x5xq5\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.242455 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.242421 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-oauth-serving-cert\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.343602 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.343535 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-console-oauth-config\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.343602 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.343572 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5xq5\" (UniqueName: \"kubernetes.io/projected/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-kube-api-access-x5xq5\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.343832 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.343769 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-oauth-serving-cert\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.343945 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.343847 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-service-ca\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.344600 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.344048 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-console-serving-cert\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.344600 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.344125 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-console-config\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.344600 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.344204 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-trusted-ca-bundle\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.344600 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.344538 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-oauth-serving-cert\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.344823 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.344710 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-service-ca\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.344892 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.344855 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-console-config\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.345300 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.345279 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-trusted-ca-bundle\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.346510 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.346493 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-console-oauth-config\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.346793 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.346774 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-console-serving-cert\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.354901 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.354856 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5xq5\" (UniqueName: \"kubernetes.io/projected/62d656ea-2f68-4fb8-8b9b-8bde992ae7d4-kube-api-access-x5xq5\") pod \"console-dd46c778-5xtfc\" (UID: \"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4\") " pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.405188 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.405161 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:02:52.573689 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:52.573657 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dd46c778-5xtfc"] Jul 24 15:02:52.574415 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:02:52.574372 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62d656ea_2f68_4fb8_8b9b_8bde992ae7d4.slice/crio-fa025f21dc8c8ccb0b0416032b46ea9326f1c719c575f0e3980131dd8fd21069 WatchSource:0}: Error finding container fa025f21dc8c8ccb0b0416032b46ea9326f1c719c575f0e3980131dd8fd21069: Status 404 returned error can't find the container with id fa025f21dc8c8ccb0b0416032b46ea9326f1c719c575f0e3980131dd8fd21069 Jul 24 15:02:53.348535 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:53.348503 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dd46c778-5xtfc" event={"ID":"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4","Type":"ContainerStarted","Data":"0df8191fc04ffc009d791c63e61efc008e420c8f2fb9e06700176e08acdfc7c3"} Jul 24 15:02:53.348917 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:53.348553 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dd46c778-5xtfc" event={"ID":"62d656ea-2f68-4fb8-8b9b-8bde992ae7d4","Type":"ContainerStarted","Data":"fa025f21dc8c8ccb0b0416032b46ea9326f1c719c575f0e3980131dd8fd21069"} Jul 24 15:02:53.370503 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:53.370450 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-dd46c778-5xtfc" podStartSLOduration=1.370433984 podStartE2EDuration="1.370433984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:02:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:02:53.368354366 +0000 UTC m=+405.598831073" watchObservedRunningTime="2026-07-24 15:02:53.370433984 +0000 UTC m=+405.600910667" Jul 24 15:02:55.229809 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:55.229768 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-xlk25" Jul 24 15:02:55.230263 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:55.229830 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-xlk25" Jul 24 15:02:55.252930 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:55.252897 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-xlk25" Jul 24 15:02:55.379852 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:55.379824 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-xlk25" Jul 24 15:02:58.129995 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:02:58.129960 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:00.217158 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:00.217126 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz"] Jul 24 15:03:00.219823 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:00.219806 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz" Jul 24 15:03:00.223146 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:00.223124 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-nd7qj\"" Jul 24 15:03:00.237288 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:00.237269 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz"] Jul 24 15:03:00.311857 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:00.311830 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8034980c-866f-4c7e-9d54-17f904645c0e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz\" (UID: \"8034980c-866f-4c7e-9d54-17f904645c0e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz" Jul 24 15:03:00.311857 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:00.311855 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8034980c-866f-4c7e-9d54-17f904645c0e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz\" (UID: \"8034980c-866f-4c7e-9d54-17f904645c0e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz" Jul 24 15:03:00.312042 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:00.311910 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcfcx\" (UniqueName: \"kubernetes.io/projected/8034980c-866f-4c7e-9d54-17f904645c0e-kube-api-access-bcfcx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz\" (UID: \"8034980c-866f-4c7e-9d54-17f904645c0e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz" Jul 24 15:03:00.413262 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:00.413235 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8034980c-866f-4c7e-9d54-17f904645c0e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz\" (UID: \"8034980c-866f-4c7e-9d54-17f904645c0e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz" Jul 24 15:03:00.413262 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:00.413266 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8034980c-866f-4c7e-9d54-17f904645c0e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz\" (UID: \"8034980c-866f-4c7e-9d54-17f904645c0e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz" Jul 24 15:03:00.413441 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:00.413305 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcfcx\" (UniqueName: \"kubernetes.io/projected/8034980c-866f-4c7e-9d54-17f904645c0e-kube-api-access-bcfcx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz\" (UID: \"8034980c-866f-4c7e-9d54-17f904645c0e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz" Jul 24 15:03:00.413612 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:00.413597 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8034980c-866f-4c7e-9d54-17f904645c0e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz\" (UID: \"8034980c-866f-4c7e-9d54-17f904645c0e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz" Jul 24 15:03:00.413653 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:00.413619 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8034980c-866f-4c7e-9d54-17f904645c0e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz\" (UID: \"8034980c-866f-4c7e-9d54-17f904645c0e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz" Jul 24 15:03:00.423402 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:00.423368 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcfcx\" (UniqueName: \"kubernetes.io/projected/8034980c-866f-4c7e-9d54-17f904645c0e-kube-api-access-bcfcx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz\" (UID: \"8034980c-866f-4c7e-9d54-17f904645c0e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz" Jul 24 15:03:00.529976 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:00.529908 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz" Jul 24 15:03:00.671047 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:00.671007 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz"] Jul 24 15:03:00.674184 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:03:00.674159 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8034980c_866f_4c7e_9d54_17f904645c0e.slice/crio-86fd66cc3a3bb68fe1715f682b2a1528e8f6d19e67e1a3b0ade8acd7ab62ddfc WatchSource:0}: Error finding container 86fd66cc3a3bb68fe1715f682b2a1528e8f6d19e67e1a3b0ade8acd7ab62ddfc: Status 404 returned error can't find the container with id 86fd66cc3a3bb68fe1715f682b2a1528e8f6d19e67e1a3b0ade8acd7ab62ddfc Jul 24 15:03:01.085209 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.085179 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7"] Jul 24 15:03:01.088044 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.088025 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7" Jul 24 15:03:01.115966 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.115943 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7"] Jul 24 15:03:01.220296 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.220269 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b653e476-5ebd-4c56-beb9-8ed2aaa66c42-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7\" (UID: \"b653e476-5ebd-4c56-beb9-8ed2aaa66c42\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7" Jul 24 15:03:01.220616 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.220310 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b653e476-5ebd-4c56-beb9-8ed2aaa66c42-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7\" (UID: \"b653e476-5ebd-4c56-beb9-8ed2aaa66c42\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7" Jul 24 15:03:01.220616 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.220380 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v2bq\" (UniqueName: \"kubernetes.io/projected/b653e476-5ebd-4c56-beb9-8ed2aaa66c42-kube-api-access-6v2bq\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7\" (UID: \"b653e476-5ebd-4c56-beb9-8ed2aaa66c42\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7" Jul 24 15:03:01.321209 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.321183 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b653e476-5ebd-4c56-beb9-8ed2aaa66c42-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7\" (UID: \"b653e476-5ebd-4c56-beb9-8ed2aaa66c42\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7" Jul 24 15:03:01.321332 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.321231 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v2bq\" (UniqueName: \"kubernetes.io/projected/b653e476-5ebd-4c56-beb9-8ed2aaa66c42-kube-api-access-6v2bq\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7\" (UID: \"b653e476-5ebd-4c56-beb9-8ed2aaa66c42\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7" Jul 24 15:03:01.321332 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.321277 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b653e476-5ebd-4c56-beb9-8ed2aaa66c42-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7\" (UID: \"b653e476-5ebd-4c56-beb9-8ed2aaa66c42\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7" Jul 24 15:03:01.321548 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.321532 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b653e476-5ebd-4c56-beb9-8ed2aaa66c42-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7\" (UID: \"b653e476-5ebd-4c56-beb9-8ed2aaa66c42\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7" Jul 24 15:03:01.321584 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.321569 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b653e476-5ebd-4c56-beb9-8ed2aaa66c42-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7\" (UID: \"b653e476-5ebd-4c56-beb9-8ed2aaa66c42\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7" Jul 24 15:03:01.332585 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.332561 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v2bq\" (UniqueName: \"kubernetes.io/projected/b653e476-5ebd-4c56-beb9-8ed2aaa66c42-kube-api-access-6v2bq\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7\" (UID: \"b653e476-5ebd-4c56-beb9-8ed2aaa66c42\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7" Jul 24 15:03:01.388785 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.388719 2531 generic.go:356] "Generic (PLEG): container finished" podID="8034980c-866f-4c7e-9d54-17f904645c0e" containerID="c27a483c2d42281b92348fa3d7b09a48992b5816ac3c3784be3babc24fdef04b" exitCode=0 Jul 24 15:03:01.388921 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.388785 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz" event={"ID":"8034980c-866f-4c7e-9d54-17f904645c0e","Type":"ContainerDied","Data":"c27a483c2d42281b92348fa3d7b09a48992b5816ac3c3784be3babc24fdef04b"} Jul 24 15:03:01.388921 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.388825 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz" event={"ID":"8034980c-866f-4c7e-9d54-17f904645c0e","Type":"ContainerStarted","Data":"86fd66cc3a3bb68fe1715f682b2a1528e8f6d19e67e1a3b0ade8acd7ab62ddfc"} Jul 24 15:03:01.397328 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.397306 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7" Jul 24 15:03:01.541648 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.541619 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7"] Jul 24 15:03:01.542988 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:03:01.542956 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb653e476_5ebd_4c56_beb9_8ed2aaa66c42.slice/crio-f2e94f358e12cd3e3e1113a6cd5cb283412843220c05b7271affa26a5a846ccd WatchSource:0}: Error finding container f2e94f358e12cd3e3e1113a6cd5cb283412843220c05b7271affa26a5a846ccd: Status 404 returned error can't find the container with id f2e94f358e12cd3e3e1113a6cd5cb283412843220c05b7271affa26a5a846ccd Jul 24 15:03:01.620925 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.620895 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm"] Jul 24 15:03:01.623880 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.623848 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm" Jul 24 15:03:01.640595 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.640532 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm"] Jul 24 15:03:01.725562 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.725530 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf08aa03-1a28-4150-9d2e-54726ea50f51-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm\" (UID: \"cf08aa03-1a28-4150-9d2e-54726ea50f51\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm" Jul 24 15:03:01.725709 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.725598 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnc2w\" (UniqueName: \"kubernetes.io/projected/cf08aa03-1a28-4150-9d2e-54726ea50f51-kube-api-access-gnc2w\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm\" (UID: \"cf08aa03-1a28-4150-9d2e-54726ea50f51\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm" Jul 24 15:03:01.725709 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.725637 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf08aa03-1a28-4150-9d2e-54726ea50f51-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm\" (UID: \"cf08aa03-1a28-4150-9d2e-54726ea50f51\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm" Jul 24 15:03:01.826840 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.826809 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf08aa03-1a28-4150-9d2e-54726ea50f51-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm\" (UID: \"cf08aa03-1a28-4150-9d2e-54726ea50f51\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm" Jul 24 15:03:01.827037 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.826887 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnc2w\" (UniqueName: \"kubernetes.io/projected/cf08aa03-1a28-4150-9d2e-54726ea50f51-kube-api-access-gnc2w\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm\" (UID: \"cf08aa03-1a28-4150-9d2e-54726ea50f51\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm" Jul 24 15:03:01.827037 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.826924 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf08aa03-1a28-4150-9d2e-54726ea50f51-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm\" (UID: \"cf08aa03-1a28-4150-9d2e-54726ea50f51\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm" Jul 24 15:03:01.827295 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.827280 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf08aa03-1a28-4150-9d2e-54726ea50f51-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm\" (UID: \"cf08aa03-1a28-4150-9d2e-54726ea50f51\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm" Jul 24 15:03:01.827336 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.827276 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf08aa03-1a28-4150-9d2e-54726ea50f51-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm\" (UID: \"cf08aa03-1a28-4150-9d2e-54726ea50f51\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm" Jul 24 15:03:01.836493 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.836468 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnc2w\" (UniqueName: \"kubernetes.io/projected/cf08aa03-1a28-4150-9d2e-54726ea50f51-kube-api-access-gnc2w\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm\" (UID: \"cf08aa03-1a28-4150-9d2e-54726ea50f51\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm" Jul 24 15:03:01.933133 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.933100 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm" Jul 24 15:03:01.976209 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.976173 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv"] Jul 24 15:03:01.979509 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.979488 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv" Jul 24 15:03:01.989073 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:01.988955 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv"] Jul 24 15:03:02.072931 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.072908 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm"] Jul 24 15:03:02.074729 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:03:02.074703 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf08aa03_1a28_4150_9d2e_54726ea50f51.slice/crio-62861b0777468a380bde6bcb7192f3dfb87636e020a24a18e4344bb9fa25f49f WatchSource:0}: Error finding container 62861b0777468a380bde6bcb7192f3dfb87636e020a24a18e4344bb9fa25f49f: Status 404 returned error can't find the container with id 62861b0777468a380bde6bcb7192f3dfb87636e020a24a18e4344bb9fa25f49f Jul 24 15:03:02.130616 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.130591 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f19e65a9-2996-41cd-89ad-80a78d4adda2-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv\" (UID: \"f19e65a9-2996-41cd-89ad-80a78d4adda2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv" Jul 24 15:03:02.130725 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.130648 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f19e65a9-2996-41cd-89ad-80a78d4adda2-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv\" (UID: \"f19e65a9-2996-41cd-89ad-80a78d4adda2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv" Jul 24 15:03:02.130725 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.130681 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzmc5\" (UniqueName: \"kubernetes.io/projected/f19e65a9-2996-41cd-89ad-80a78d4adda2-kube-api-access-fzmc5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv\" (UID: \"f19e65a9-2996-41cd-89ad-80a78d4adda2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv" Jul 24 15:03:02.232107 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.232020 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f19e65a9-2996-41cd-89ad-80a78d4adda2-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv\" (UID: \"f19e65a9-2996-41cd-89ad-80a78d4adda2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv" Jul 24 15:03:02.232107 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.232077 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f19e65a9-2996-41cd-89ad-80a78d4adda2-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv\" (UID: \"f19e65a9-2996-41cd-89ad-80a78d4adda2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv" Jul 24 15:03:02.232568 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.232107 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzmc5\" (UniqueName: \"kubernetes.io/projected/f19e65a9-2996-41cd-89ad-80a78d4adda2-kube-api-access-fzmc5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv\" (UID: \"f19e65a9-2996-41cd-89ad-80a78d4adda2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv" Jul 24 15:03:02.232568 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.232438 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f19e65a9-2996-41cd-89ad-80a78d4adda2-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv\" (UID: \"f19e65a9-2996-41cd-89ad-80a78d4adda2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv" Jul 24 15:03:02.232568 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.232478 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f19e65a9-2996-41cd-89ad-80a78d4adda2-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv\" (UID: \"f19e65a9-2996-41cd-89ad-80a78d4adda2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv" Jul 24 15:03:02.241107 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.241089 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzmc5\" (UniqueName: \"kubernetes.io/projected/f19e65a9-2996-41cd-89ad-80a78d4adda2-kube-api-access-fzmc5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv\" (UID: \"f19e65a9-2996-41cd-89ad-80a78d4adda2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv" Jul 24 15:03:02.297212 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.297186 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv" Jul 24 15:03:02.397058 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.397027 2531 generic.go:356] "Generic (PLEG): container finished" podID="b653e476-5ebd-4c56-beb9-8ed2aaa66c42" containerID="a5637a0d04c16fd202668b80ae9a6d282a76de284cf63686f7c92ea4b816b567" exitCode=0 Jul 24 15:03:02.397206 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.397111 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7" event={"ID":"b653e476-5ebd-4c56-beb9-8ed2aaa66c42","Type":"ContainerDied","Data":"a5637a0d04c16fd202668b80ae9a6d282a76de284cf63686f7c92ea4b816b567"} Jul 24 15:03:02.397206 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.397167 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7" event={"ID":"b653e476-5ebd-4c56-beb9-8ed2aaa66c42","Type":"ContainerStarted","Data":"f2e94f358e12cd3e3e1113a6cd5cb283412843220c05b7271affa26a5a846ccd"} Jul 24 15:03:02.399132 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.399107 2531 generic.go:356] "Generic (PLEG): container finished" podID="cf08aa03-1a28-4150-9d2e-54726ea50f51" containerID="e6032a942ce8a64c1137c09dd8d07e0a37ec0a8b30bc6538ef8b31ebdba26bd1" exitCode=0 Jul 24 15:03:02.399251 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.399198 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm" event={"ID":"cf08aa03-1a28-4150-9d2e-54726ea50f51","Type":"ContainerDied","Data":"e6032a942ce8a64c1137c09dd8d07e0a37ec0a8b30bc6538ef8b31ebdba26bd1"} Jul 24 15:03:02.399294 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.399258 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm" event={"ID":"cf08aa03-1a28-4150-9d2e-54726ea50f51","Type":"ContainerStarted","Data":"62861b0777468a380bde6bcb7192f3dfb87636e020a24a18e4344bb9fa25f49f"} Jul 24 15:03:02.405379 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.405355 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:03:02.405645 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.405628 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:03:02.410619 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.410597 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:03:02.440592 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:02.440575 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv"] Jul 24 15:03:02.442997 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:03:02.442968 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf19e65a9_2996_41cd_89ad_80a78d4adda2.slice/crio-530816de1ddf3f7ca2446500a8b280c387aa55974421ed8d4a0944efaf03d9e0 WatchSource:0}: Error finding container 530816de1ddf3f7ca2446500a8b280c387aa55974421ed8d4a0944efaf03d9e0: Status 404 returned error can't find the container with id 530816de1ddf3f7ca2446500a8b280c387aa55974421ed8d4a0944efaf03d9e0 Jul 24 15:03:03.405932 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:03.405901 2531 generic.go:356] "Generic (PLEG): container finished" podID="8034980c-866f-4c7e-9d54-17f904645c0e" containerID="ca6ca6339b0483b987c6eca7ebbc0f552c3559da587dcd6459a2240b111231c2" exitCode=0 Jul 24 15:03:03.406227 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:03.405982 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz" event={"ID":"8034980c-866f-4c7e-9d54-17f904645c0e","Type":"ContainerDied","Data":"ca6ca6339b0483b987c6eca7ebbc0f552c3559da587dcd6459a2240b111231c2"} Jul 24 15:03:03.407586 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:03.407562 2531 generic.go:356] "Generic (PLEG): container finished" podID="f19e65a9-2996-41cd-89ad-80a78d4adda2" containerID="2d586ec997d79c9668a5198b150d89582f6869195384a8f985f035c4435621c3" exitCode=0 Jul 24 15:03:03.407679 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:03.407648 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv" event={"ID":"f19e65a9-2996-41cd-89ad-80a78d4adda2","Type":"ContainerDied","Data":"2d586ec997d79c9668a5198b150d89582f6869195384a8f985f035c4435621c3"} Jul 24 15:03:03.407743 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:03.407689 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv" event={"ID":"f19e65a9-2996-41cd-89ad-80a78d4adda2","Type":"ContainerStarted","Data":"530816de1ddf3f7ca2446500a8b280c387aa55974421ed8d4a0944efaf03d9e0"} Jul 24 15:03:03.412447 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:03.412428 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-dd46c778-5xtfc" Jul 24 15:03:03.494527 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:03.494483 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b588b9796-8vjfp"] Jul 24 15:03:03.652769 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:03.652738 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:03.966727 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:03.966694 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:04.046062 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:04.046032 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:04.414858 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:04.414775 2531 generic.go:356] "Generic (PLEG): container finished" podID="b653e476-5ebd-4c56-beb9-8ed2aaa66c42" containerID="e0406ee0cda64af8e6f43d25ad8eeb17473f95a59864d43402efde4e8db8bfb5" exitCode=0 Jul 24 15:03:04.415324 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:04.414875 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7" event={"ID":"b653e476-5ebd-4c56-beb9-8ed2aaa66c42","Type":"ContainerDied","Data":"e0406ee0cda64af8e6f43d25ad8eeb17473f95a59864d43402efde4e8db8bfb5"} Jul 24 15:03:04.417198 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:04.417176 2531 generic.go:356] "Generic (PLEG): container finished" podID="8034980c-866f-4c7e-9d54-17f904645c0e" containerID="00b0d7531b23e53bcf6ee1bd3a2b33c27392e62bc5295cbbe0b5930a8ca6e9b1" exitCode=0 Jul 24 15:03:04.417309 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:04.417257 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz" event={"ID":"8034980c-866f-4c7e-9d54-17f904645c0e","Type":"ContainerDied","Data":"00b0d7531b23e53bcf6ee1bd3a2b33c27392e62bc5295cbbe0b5930a8ca6e9b1"} Jul 24 15:03:04.419031 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:04.418994 2531 generic.go:356] "Generic (PLEG): container finished" podID="cf08aa03-1a28-4150-9d2e-54726ea50f51" containerID="33430e9d4774257b6024e7c5dd36337636150e339f1833d732d6eefa7e43e175" exitCode=0 Jul 24 15:03:04.419131 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:04.419039 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm" event={"ID":"cf08aa03-1a28-4150-9d2e-54726ea50f51","Type":"ContainerDied","Data":"33430e9d4774257b6024e7c5dd36337636150e339f1833d732d6eefa7e43e175"} Jul 24 15:03:05.426483 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:05.426452 2531 generic.go:356] "Generic (PLEG): container finished" podID="cf08aa03-1a28-4150-9d2e-54726ea50f51" containerID="ff56db9d2dbc6e1b168efdfef68ba329566ed5753e0008e533c1e0def3a14292" exitCode=0 Jul 24 15:03:05.426910 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:05.426534 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm" event={"ID":"cf08aa03-1a28-4150-9d2e-54726ea50f51","Type":"ContainerDied","Data":"ff56db9d2dbc6e1b168efdfef68ba329566ed5753e0008e533c1e0def3a14292"} Jul 24 15:03:05.428189 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:05.428168 2531 generic.go:356] "Generic (PLEG): container finished" podID="f19e65a9-2996-41cd-89ad-80a78d4adda2" containerID="2a01152c9a9931bfd5edae70c0c8664fd8e3b3a3af61c6cfbcda35a2a2baa2ac" exitCode=0 Jul 24 15:03:05.428309 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:05.428245 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv" event={"ID":"f19e65a9-2996-41cd-89ad-80a78d4adda2","Type":"ContainerDied","Data":"2a01152c9a9931bfd5edae70c0c8664fd8e3b3a3af61c6cfbcda35a2a2baa2ac"} Jul 24 15:03:05.430567 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:05.430001 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:05.430973 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:05.430951 2531 generic.go:356] "Generic (PLEG): container finished" podID="b653e476-5ebd-4c56-beb9-8ed2aaa66c42" containerID="1fe71a35428d54c9b5d20373245848477d91886b129f5f080e4db86de8547113" exitCode=0 Jul 24 15:03:05.431082 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:05.431040 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7" event={"ID":"b653e476-5ebd-4c56-beb9-8ed2aaa66c42","Type":"ContainerDied","Data":"1fe71a35428d54c9b5d20373245848477d91886b129f5f080e4db86de8547113"} Jul 24 15:03:05.574760 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:05.574734 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz" Jul 24 15:03:05.766075 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:05.765999 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcfcx\" (UniqueName: \"kubernetes.io/projected/8034980c-866f-4c7e-9d54-17f904645c0e-kube-api-access-bcfcx\") pod \"8034980c-866f-4c7e-9d54-17f904645c0e\" (UID: \"8034980c-866f-4c7e-9d54-17f904645c0e\") " Jul 24 15:03:05.766201 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:05.766114 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8034980c-866f-4c7e-9d54-17f904645c0e-util\") pod \"8034980c-866f-4c7e-9d54-17f904645c0e\" (UID: \"8034980c-866f-4c7e-9d54-17f904645c0e\") " Jul 24 15:03:05.766201 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:05.766155 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8034980c-866f-4c7e-9d54-17f904645c0e-bundle\") pod \"8034980c-866f-4c7e-9d54-17f904645c0e\" (UID: \"8034980c-866f-4c7e-9d54-17f904645c0e\") " Jul 24 15:03:05.766672 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:05.766645 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8034980c-866f-4c7e-9d54-17f904645c0e-bundle" (OuterVolumeSpecName: "bundle") pod "8034980c-866f-4c7e-9d54-17f904645c0e" (UID: "8034980c-866f-4c7e-9d54-17f904645c0e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:03:05.768185 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:05.768157 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8034980c-866f-4c7e-9d54-17f904645c0e-kube-api-access-bcfcx" (OuterVolumeSpecName: "kube-api-access-bcfcx") pod "8034980c-866f-4c7e-9d54-17f904645c0e" (UID: "8034980c-866f-4c7e-9d54-17f904645c0e"). InnerVolumeSpecName "kube-api-access-bcfcx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:03:05.772103 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:05.772084 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8034980c-866f-4c7e-9d54-17f904645c0e-util" (OuterVolumeSpecName: "util") pod "8034980c-866f-4c7e-9d54-17f904645c0e" (UID: "8034980c-866f-4c7e-9d54-17f904645c0e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:03:05.867209 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:05.867182 2531 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8034980c-866f-4c7e-9d54-17f904645c0e-util\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:05.867209 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:05.867205 2531 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8034980c-866f-4c7e-9d54-17f904645c0e-bundle\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:05.867347 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:05.867214 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcfcx\" (UniqueName: \"kubernetes.io/projected/8034980c-866f-4c7e-9d54-17f904645c0e-kube-api-access-bcfcx\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:06.438233 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.438205 2531 generic.go:356] "Generic (PLEG): container finished" podID="f19e65a9-2996-41cd-89ad-80a78d4adda2" containerID="928c12e958fba4aec190a5d21c17bd58d068f9931b20d8e7f3b36aa4d15ebc15" exitCode=0 Jul 24 15:03:06.438595 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.438290 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv" event={"ID":"f19e65a9-2996-41cd-89ad-80a78d4adda2","Type":"ContainerDied","Data":"928c12e958fba4aec190a5d21c17bd58d068f9931b20d8e7f3b36aa4d15ebc15"} Jul 24 15:03:06.440324 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.440302 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz" event={"ID":"8034980c-866f-4c7e-9d54-17f904645c0e","Type":"ContainerDied","Data":"86fd66cc3a3bb68fe1715f682b2a1528e8f6d19e67e1a3b0ade8acd7ab62ddfc"} Jul 24 15:03:06.440324 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.440319 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz" Jul 24 15:03:06.440509 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.440340 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86fd66cc3a3bb68fe1715f682b2a1528e8f6d19e67e1a3b0ade8acd7ab62ddfc" Jul 24 15:03:06.583196 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.583171 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm" Jul 24 15:03:06.617999 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.617972 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7" Jul 24 15:03:06.776472 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.776435 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b653e476-5ebd-4c56-beb9-8ed2aaa66c42-util\") pod \"b653e476-5ebd-4c56-beb9-8ed2aaa66c42\" (UID: \"b653e476-5ebd-4c56-beb9-8ed2aaa66c42\") " Jul 24 15:03:06.776472 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.776480 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnc2w\" (UniqueName: \"kubernetes.io/projected/cf08aa03-1a28-4150-9d2e-54726ea50f51-kube-api-access-gnc2w\") pod \"cf08aa03-1a28-4150-9d2e-54726ea50f51\" (UID: \"cf08aa03-1a28-4150-9d2e-54726ea50f51\") " Jul 24 15:03:06.776725 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.776518 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b653e476-5ebd-4c56-beb9-8ed2aaa66c42-bundle\") pod \"b653e476-5ebd-4c56-beb9-8ed2aaa66c42\" (UID: \"b653e476-5ebd-4c56-beb9-8ed2aaa66c42\") " Jul 24 15:03:06.776725 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.776538 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf08aa03-1a28-4150-9d2e-54726ea50f51-bundle\") pod \"cf08aa03-1a28-4150-9d2e-54726ea50f51\" (UID: \"cf08aa03-1a28-4150-9d2e-54726ea50f51\") " Jul 24 15:03:06.776725 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.776570 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v2bq\" (UniqueName: \"kubernetes.io/projected/b653e476-5ebd-4c56-beb9-8ed2aaa66c42-kube-api-access-6v2bq\") pod \"b653e476-5ebd-4c56-beb9-8ed2aaa66c42\" (UID: \"b653e476-5ebd-4c56-beb9-8ed2aaa66c42\") " Jul 24 15:03:06.776725 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.776616 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf08aa03-1a28-4150-9d2e-54726ea50f51-util\") pod \"cf08aa03-1a28-4150-9d2e-54726ea50f51\" (UID: \"cf08aa03-1a28-4150-9d2e-54726ea50f51\") " Jul 24 15:03:06.777083 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.777055 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b653e476-5ebd-4c56-beb9-8ed2aaa66c42-bundle" (OuterVolumeSpecName: "bundle") pod "b653e476-5ebd-4c56-beb9-8ed2aaa66c42" (UID: "b653e476-5ebd-4c56-beb9-8ed2aaa66c42"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:03:06.777083 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.777059 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf08aa03-1a28-4150-9d2e-54726ea50f51-bundle" (OuterVolumeSpecName: "bundle") pod "cf08aa03-1a28-4150-9d2e-54726ea50f51" (UID: "cf08aa03-1a28-4150-9d2e-54726ea50f51"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:03:06.778776 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.778744 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf08aa03-1a28-4150-9d2e-54726ea50f51-kube-api-access-gnc2w" (OuterVolumeSpecName: "kube-api-access-gnc2w") pod "cf08aa03-1a28-4150-9d2e-54726ea50f51" (UID: "cf08aa03-1a28-4150-9d2e-54726ea50f51"). InnerVolumeSpecName "kube-api-access-gnc2w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:03:06.778950 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.778776 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b653e476-5ebd-4c56-beb9-8ed2aaa66c42-kube-api-access-6v2bq" (OuterVolumeSpecName: "kube-api-access-6v2bq") pod "b653e476-5ebd-4c56-beb9-8ed2aaa66c42" (UID: "b653e476-5ebd-4c56-beb9-8ed2aaa66c42"). InnerVolumeSpecName "kube-api-access-6v2bq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:03:06.782310 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.782274 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf08aa03-1a28-4150-9d2e-54726ea50f51-util" (OuterVolumeSpecName: "util") pod "cf08aa03-1a28-4150-9d2e-54726ea50f51" (UID: "cf08aa03-1a28-4150-9d2e-54726ea50f51"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:03:06.782528 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.782511 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b653e476-5ebd-4c56-beb9-8ed2aaa66c42-util" (OuterVolumeSpecName: "util") pod "b653e476-5ebd-4c56-beb9-8ed2aaa66c42" (UID: "b653e476-5ebd-4c56-beb9-8ed2aaa66c42"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:03:06.877312 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.877241 2531 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b653e476-5ebd-4c56-beb9-8ed2aaa66c42-util\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:06.877312 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.877263 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnc2w\" (UniqueName: \"kubernetes.io/projected/cf08aa03-1a28-4150-9d2e-54726ea50f51-kube-api-access-gnc2w\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:06.877312 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.877274 2531 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b653e476-5ebd-4c56-beb9-8ed2aaa66c42-bundle\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:06.877312 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.877283 2531 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf08aa03-1a28-4150-9d2e-54726ea50f51-bundle\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:06.877312 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.877291 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v2bq\" (UniqueName: \"kubernetes.io/projected/b653e476-5ebd-4c56-beb9-8ed2aaa66c42-kube-api-access-6v2bq\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:06.877312 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:06.877298 2531 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf08aa03-1a28-4150-9d2e-54726ea50f51-util\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:07.447615 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:07.447591 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm" Jul 24 15:03:07.448019 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:07.447627 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm" event={"ID":"cf08aa03-1a28-4150-9d2e-54726ea50f51","Type":"ContainerDied","Data":"62861b0777468a380bde6bcb7192f3dfb87636e020a24a18e4344bb9fa25f49f"} Jul 24 15:03:07.448019 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:07.447675 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62861b0777468a380bde6bcb7192f3dfb87636e020a24a18e4344bb9fa25f49f" Jul 24 15:03:07.449569 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:07.449545 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7" Jul 24 15:03:07.449665 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:07.449565 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7" event={"ID":"b653e476-5ebd-4c56-beb9-8ed2aaa66c42","Type":"ContainerDied","Data":"f2e94f358e12cd3e3e1113a6cd5cb283412843220c05b7271affa26a5a846ccd"} Jul 24 15:03:07.449665 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:07.449602 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2e94f358e12cd3e3e1113a6cd5cb283412843220c05b7271affa26a5a846ccd" Jul 24 15:03:07.600023 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:07.599994 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv" Jul 24 15:03:07.786086 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:07.786011 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f19e65a9-2996-41cd-89ad-80a78d4adda2-bundle\") pod \"f19e65a9-2996-41cd-89ad-80a78d4adda2\" (UID: \"f19e65a9-2996-41cd-89ad-80a78d4adda2\") " Jul 24 15:03:07.786228 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:07.786114 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f19e65a9-2996-41cd-89ad-80a78d4adda2-util\") pod \"f19e65a9-2996-41cd-89ad-80a78d4adda2\" (UID: \"f19e65a9-2996-41cd-89ad-80a78d4adda2\") " Jul 24 15:03:07.786228 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:07.786156 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzmc5\" (UniqueName: \"kubernetes.io/projected/f19e65a9-2996-41cd-89ad-80a78d4adda2-kube-api-access-fzmc5\") pod \"f19e65a9-2996-41cd-89ad-80a78d4adda2\" (UID: \"f19e65a9-2996-41cd-89ad-80a78d4adda2\") " Jul 24 15:03:07.786667 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:07.786642 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f19e65a9-2996-41cd-89ad-80a78d4adda2-bundle" (OuterVolumeSpecName: "bundle") pod "f19e65a9-2996-41cd-89ad-80a78d4adda2" (UID: "f19e65a9-2996-41cd-89ad-80a78d4adda2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:03:07.788346 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:07.788325 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f19e65a9-2996-41cd-89ad-80a78d4adda2-kube-api-access-fzmc5" (OuterVolumeSpecName: "kube-api-access-fzmc5") pod "f19e65a9-2996-41cd-89ad-80a78d4adda2" (UID: "f19e65a9-2996-41cd-89ad-80a78d4adda2"). InnerVolumeSpecName "kube-api-access-fzmc5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:03:07.790826 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:07.790783 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f19e65a9-2996-41cd-89ad-80a78d4adda2-util" (OuterVolumeSpecName: "util") pod "f19e65a9-2996-41cd-89ad-80a78d4adda2" (UID: "f19e65a9-2996-41cd-89ad-80a78d4adda2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:03:07.887761 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:07.887735 2531 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f19e65a9-2996-41cd-89ad-80a78d4adda2-bundle\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:07.887761 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:07.887757 2531 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f19e65a9-2996-41cd-89ad-80a78d4adda2-util\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:07.887923 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:07.887767 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzmc5\" (UniqueName: \"kubernetes.io/projected/f19e65a9-2996-41cd-89ad-80a78d4adda2-kube-api-access-fzmc5\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:08.456658 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:08.456573 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv" Jul 24 15:03:08.456658 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:08.456599 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv" event={"ID":"f19e65a9-2996-41cd-89ad-80a78d4adda2","Type":"ContainerDied","Data":"530816de1ddf3f7ca2446500a8b280c387aa55974421ed8d4a0944efaf03d9e0"} Jul 24 15:03:08.456658 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:08.456648 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="530816de1ddf3f7ca2446500a8b280c387aa55974421ed8d4a0944efaf03d9e0" Jul 24 15:03:13.719202 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:13.719171 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:17.394829 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:17.394801 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:18.564467 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.564428 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx"] Jul 24 15:03:18.565001 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.564983 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b653e476-5ebd-4c56-beb9-8ed2aaa66c42" containerName="pull" Jul 24 15:03:18.565079 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565004 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="b653e476-5ebd-4c56-beb9-8ed2aaa66c42" containerName="pull" Jul 24 15:03:18.565079 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565021 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8034980c-866f-4c7e-9d54-17f904645c0e" containerName="util" Jul 24 15:03:18.565079 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565030 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="8034980c-866f-4c7e-9d54-17f904645c0e" containerName="util" Jul 24 15:03:18.565079 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565042 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8034980c-866f-4c7e-9d54-17f904645c0e" containerName="pull" Jul 24 15:03:18.565079 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565051 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="8034980c-866f-4c7e-9d54-17f904645c0e" containerName="pull" Jul 24 15:03:18.565079 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565060 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf08aa03-1a28-4150-9d2e-54726ea50f51" containerName="extract" Jul 24 15:03:18.565079 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565070 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf08aa03-1a28-4150-9d2e-54726ea50f51" containerName="extract" Jul 24 15:03:18.565079 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565080 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f19e65a9-2996-41cd-89ad-80a78d4adda2" containerName="pull" Jul 24 15:03:18.565459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565088 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="f19e65a9-2996-41cd-89ad-80a78d4adda2" containerName="pull" Jul 24 15:03:18.565459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565098 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f19e65a9-2996-41cd-89ad-80a78d4adda2" containerName="extract" Jul 24 15:03:18.565459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565106 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="f19e65a9-2996-41cd-89ad-80a78d4adda2" containerName="extract" Jul 24 15:03:18.565459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565134 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8034980c-866f-4c7e-9d54-17f904645c0e" containerName="extract" Jul 24 15:03:18.565459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565144 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="8034980c-866f-4c7e-9d54-17f904645c0e" containerName="extract" Jul 24 15:03:18.565459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565162 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf08aa03-1a28-4150-9d2e-54726ea50f51" containerName="pull" Jul 24 15:03:18.565459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565169 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf08aa03-1a28-4150-9d2e-54726ea50f51" containerName="pull" Jul 24 15:03:18.565459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565180 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b653e476-5ebd-4c56-beb9-8ed2aaa66c42" containerName="util" Jul 24 15:03:18.565459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565187 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="b653e476-5ebd-4c56-beb9-8ed2aaa66c42" containerName="util" Jul 24 15:03:18.565459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565196 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f19e65a9-2996-41cd-89ad-80a78d4adda2" containerName="util" Jul 24 15:03:18.565459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565203 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="f19e65a9-2996-41cd-89ad-80a78d4adda2" containerName="util" Jul 24 15:03:18.565459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565212 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b653e476-5ebd-4c56-beb9-8ed2aaa66c42" containerName="extract" Jul 24 15:03:18.565459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565220 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="b653e476-5ebd-4c56-beb9-8ed2aaa66c42" containerName="extract" Jul 24 15:03:18.565459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565232 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf08aa03-1a28-4150-9d2e-54726ea50f51" containerName="util" Jul 24 15:03:18.565459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565240 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf08aa03-1a28-4150-9d2e-54726ea50f51" containerName="util" Jul 24 15:03:18.565459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565337 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="b653e476-5ebd-4c56-beb9-8ed2aaa66c42" containerName="extract" Jul 24 15:03:18.565459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565351 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf08aa03-1a28-4150-9d2e-54726ea50f51" containerName="extract" Jul 24 15:03:18.565459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565361 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="8034980c-866f-4c7e-9d54-17f904645c0e" containerName="extract" Jul 24 15:03:18.565459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.565376 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="f19e65a9-2996-41cd-89ad-80a78d4adda2" containerName="extract" Jul 24 15:03:18.570393 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.570371 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" Jul 24 15:03:18.574689 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.574672 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-sf7fs\"" Jul 24 15:03:18.584998 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.584973 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx"] Jul 24 15:03:18.682662 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.682622 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhh2k\" (UniqueName: \"kubernetes.io/projected/58e333e1-d447-4c42-9c8e-158ddd899b06-kube-api-access-xhh2k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6wnxx\" (UID: \"58e333e1-d447-4c42-9c8e-158ddd899b06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" Jul 24 15:03:18.682824 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.682668 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58e333e1-d447-4c42-9c8e-158ddd899b06-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6wnxx\" (UID: \"58e333e1-d447-4c42-9c8e-158ddd899b06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" Jul 24 15:03:18.738123 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.738081 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:18.784079 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.784053 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhh2k\" (UniqueName: \"kubernetes.io/projected/58e333e1-d447-4c42-9c8e-158ddd899b06-kube-api-access-xhh2k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6wnxx\" (UID: \"58e333e1-d447-4c42-9c8e-158ddd899b06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" Jul 24 15:03:18.784181 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.784091 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58e333e1-d447-4c42-9c8e-158ddd899b06-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6wnxx\" (UID: \"58e333e1-d447-4c42-9c8e-158ddd899b06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" Jul 24 15:03:18.784396 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.784382 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58e333e1-d447-4c42-9c8e-158ddd899b06-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6wnxx\" (UID: \"58e333e1-d447-4c42-9c8e-158ddd899b06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" Jul 24 15:03:18.797068 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.797048 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhh2k\" (UniqueName: \"kubernetes.io/projected/58e333e1-d447-4c42-9c8e-158ddd899b06-kube-api-access-xhh2k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6wnxx\" (UID: \"58e333e1-d447-4c42-9c8e-158ddd899b06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" Jul 24 15:03:18.880699 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:18.880629 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" Jul 24 15:03:19.021305 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:19.021282 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx"] Jul 24 15:03:19.022920 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:03:19.022891 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58e333e1_d447_4c42_9c8e_158ddd899b06.slice/crio-06360d0b2b6dd85db76ea2d2de7fb04401117484d20602ec2cb0909e77c6a0e3 WatchSource:0}: Error finding container 06360d0b2b6dd85db76ea2d2de7fb04401117484d20602ec2cb0909e77c6a0e3: Status 404 returned error can't find the container with id 06360d0b2b6dd85db76ea2d2de7fb04401117484d20602ec2cb0909e77c6a0e3 Jul 24 15:03:19.122366 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:19.122333 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:19.516153 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:19.516121 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" event={"ID":"58e333e1-d447-4c42-9c8e-158ddd899b06","Type":"ContainerStarted","Data":"06360d0b2b6dd85db76ea2d2de7fb04401117484d20602ec2cb0909e77c6a0e3"} Jul 24 15:03:20.208409 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:20.208362 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:20.315315 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:20.315280 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:21.223710 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:21.223673 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-s5kck"] Jul 24 15:03:21.231510 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:21.231494 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s5kck" Jul 24 15:03:21.234996 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:21.234979 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-brzfv\"" Jul 24 15:03:21.251376 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:21.251345 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-s5kck"] Jul 24 15:03:21.309368 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:21.309334 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjjqg\" (UniqueName: \"kubernetes.io/projected/3504103a-4dcf-400b-a0a5-578d2c374fd2-kube-api-access-sjjqg\") pod \"authorino-operator-75cc8d5cb7-s5kck\" (UID: \"3504103a-4dcf-400b-a0a5-578d2c374fd2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-s5kck" Jul 24 15:03:21.410606 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:21.410568 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjjqg\" (UniqueName: \"kubernetes.io/projected/3504103a-4dcf-400b-a0a5-578d2c374fd2-kube-api-access-sjjqg\") pod \"authorino-operator-75cc8d5cb7-s5kck\" (UID: \"3504103a-4dcf-400b-a0a5-578d2c374fd2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-s5kck" Jul 24 15:03:21.420974 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:21.420940 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjjqg\" (UniqueName: \"kubernetes.io/projected/3504103a-4dcf-400b-a0a5-578d2c374fd2-kube-api-access-sjjqg\") pod \"authorino-operator-75cc8d5cb7-s5kck\" (UID: \"3504103a-4dcf-400b-a0a5-578d2c374fd2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-s5kck" Jul 24 15:03:21.541858 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:21.541771 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s5kck" Jul 24 15:03:21.715171 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:21.715060 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-s5kck"] Jul 24 15:03:21.717611 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:03:21.717582 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3504103a_4dcf_400b_a0a5_578d2c374fd2.slice/crio-c577d1423ba0da481bb714e45e1f8b7a21813d6cdf9a08e4ebe5cbbfd7b02c03 WatchSource:0}: Error finding container c577d1423ba0da481bb714e45e1f8b7a21813d6cdf9a08e4ebe5cbbfd7b02c03: Status 404 returned error can't find the container with id c577d1423ba0da481bb714e45e1f8b7a21813d6cdf9a08e4ebe5cbbfd7b02c03 Jul 24 15:03:21.892146 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:21.892036 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:22.026498 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:22.026468 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:22.536005 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:22.535971 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s5kck" event={"ID":"3504103a-4dcf-400b-a0a5-578d2c374fd2","Type":"ContainerStarted","Data":"c577d1423ba0da481bb714e45e1f8b7a21813d6cdf9a08e4ebe5cbbfd7b02c03"} Jul 24 15:03:23.379447 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:23.379386 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-kx664"] Jul 24 15:03:23.383336 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:23.383313 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kx664" Jul 24 15:03:23.386635 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:23.386459 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 24 15:03:23.386635 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:23.386529 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-r6fvm\"" Jul 24 15:03:23.399002 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:23.398979 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-kx664"] Jul 24 15:03:23.438535 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:23.438508 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pvbq\" (UniqueName: \"kubernetes.io/projected/5a357ffd-7231-4fb2-8fec-3b050c5a64bc-kube-api-access-9pvbq\") pod \"dns-operator-controller-manager-864dd78c4c-kx664\" (UID: \"5a357ffd-7231-4fb2-8fec-3b050c5a64bc\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kx664" Jul 24 15:03:23.539631 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:23.539596 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pvbq\" (UniqueName: \"kubernetes.io/projected/5a357ffd-7231-4fb2-8fec-3b050c5a64bc-kube-api-access-9pvbq\") pod \"dns-operator-controller-manager-864dd78c4c-kx664\" (UID: \"5a357ffd-7231-4fb2-8fec-3b050c5a64bc\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kx664" Jul 24 15:03:23.553734 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:23.553704 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pvbq\" (UniqueName: \"kubernetes.io/projected/5a357ffd-7231-4fb2-8fec-3b050c5a64bc-kube-api-access-9pvbq\") pod \"dns-operator-controller-manager-864dd78c4c-kx664\" (UID: \"5a357ffd-7231-4fb2-8fec-3b050c5a64bc\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kx664" Jul 24 15:03:23.699440 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:23.699387 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kx664" Jul 24 15:03:23.931938 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:23.931836 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:25.418517 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:25.418484 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r"] Jul 24 15:03:25.422451 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:25.422423 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r" Jul 24 15:03:25.425117 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:25.425095 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-ffwrh\"" Jul 24 15:03:25.433467 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:25.433405 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r"] Jul 24 15:03:25.561976 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:25.561937 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5qkm\" (UniqueName: \"kubernetes.io/projected/7e1ea8a8-9a24-48b0-910b-d43f2e8c951b-kube-api-access-h5qkm\") pod \"limitador-operator-controller-manager-859d7b78bd-pf47r\" (UID: \"7e1ea8a8-9a24-48b0-910b-d43f2e8c951b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r" Jul 24 15:03:25.662488 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:25.662451 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5qkm\" (UniqueName: \"kubernetes.io/projected/7e1ea8a8-9a24-48b0-910b-d43f2e8c951b-kube-api-access-h5qkm\") pod \"limitador-operator-controller-manager-859d7b78bd-pf47r\" (UID: \"7e1ea8a8-9a24-48b0-910b-d43f2e8c951b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r" Jul 24 15:03:25.683642 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:25.683613 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5qkm\" (UniqueName: \"kubernetes.io/projected/7e1ea8a8-9a24-48b0-910b-d43f2e8c951b-kube-api-access-h5qkm\") pod \"limitador-operator-controller-manager-859d7b78bd-pf47r\" (UID: \"7e1ea8a8-9a24-48b0-910b-d43f2e8c951b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r" Jul 24 15:03:25.736417 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:25.736193 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r" Jul 24 15:03:25.784970 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:25.784944 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:25.855442 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:25.855403 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-kx664"] Jul 24 15:03:25.858146 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:03:25.858111 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a357ffd_7231_4fb2_8fec_3b050c5a64bc.slice/crio-6198b8ece8721aa6f6520141cfc3fe8a5c3dd7242a5a86dcac8882233d0c9074 WatchSource:0}: Error finding container 6198b8ece8721aa6f6520141cfc3fe8a5c3dd7242a5a86dcac8882233d0c9074: Status 404 returned error can't find the container with id 6198b8ece8721aa6f6520141cfc3fe8a5c3dd7242a5a86dcac8882233d0c9074 Jul 24 15:03:25.900010 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:25.899972 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r"] Jul 24 15:03:25.902604 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:03:25.902580 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e1ea8a8_9a24_48b0_910b_d43f2e8c951b.slice/crio-f1d7b9832b7d12484a0c00caa58e600a4fa9f2c5d18973db50b3fad6559dc738 WatchSource:0}: Error finding container f1d7b9832b7d12484a0c00caa58e600a4fa9f2c5d18973db50b3fad6559dc738: Status 404 returned error can't find the container with id f1d7b9832b7d12484a0c00caa58e600a4fa9f2c5d18973db50b3fad6559dc738 Jul 24 15:03:26.573394 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:26.573352 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s5kck" event={"ID":"3504103a-4dcf-400b-a0a5-578d2c374fd2","Type":"ContainerStarted","Data":"783578886ee1cd0d7277791a7199b9210d63b5a7cb5c9621376275106a5201fd"} Jul 24 15:03:26.573840 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:26.573472 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s5kck" Jul 24 15:03:26.575171 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:26.575136 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kx664" event={"ID":"5a357ffd-7231-4fb2-8fec-3b050c5a64bc","Type":"ContainerStarted","Data":"6198b8ece8721aa6f6520141cfc3fe8a5c3dd7242a5a86dcac8882233d0c9074"} Jul 24 15:03:26.577628 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:26.577604 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" event={"ID":"58e333e1-d447-4c42-9c8e-158ddd899b06","Type":"ContainerStarted","Data":"3cb889eb7243e18f6f41066054b5d831a381cc869a4e0bda07396a7457fd5ab5"} Jul 24 15:03:26.577712 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:26.577659 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" Jul 24 15:03:26.579131 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:26.579109 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r" event={"ID":"7e1ea8a8-9a24-48b0-910b-d43f2e8c951b","Type":"ContainerStarted","Data":"f1d7b9832b7d12484a0c00caa58e600a4fa9f2c5d18973db50b3fad6559dc738"} Jul 24 15:03:26.598984 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:26.598933 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s5kck" podStartSLOduration=1.6257440810000001 podStartE2EDuration="5.598913009s" totalImagesPullingTime="3.973168928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:03:26.595835999 +0000 UTC m=+438.826312711" watchObservedRunningTime="2026-07-24 15:03:26.598913009 +0000 UTC m=+438.829389708" Jul 24 15:03:26.621918 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:26.621847 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" podStartSLOduration=1.940746369 podStartE2EDuration="8.621826294s" totalImagesPullingTime="6.681079925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:03:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:03:26.618623115 +0000 UTC m=+438.849099827" watchObservedRunningTime="2026-07-24 15:03:26.621826294 +0000 UTC m=+438.852302982" Jul 24 15:03:27.546349 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:27.546292 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:27.641949 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:27.641915 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:28.519904 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.519845 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5b588b9796-8vjfp" podUID="2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b" containerName="console" containerID="cri-o://7d5dc93bb81e4000ccb957598ac4902ff410bba13c07d4a941531a3e85341eac" gracePeriod=15 Jul 24 15:03:28.594269 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.594196 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r" event={"ID":"7e1ea8a8-9a24-48b0-910b-d43f2e8c951b","Type":"ContainerStarted","Data":"5556922f1174f17080cdf16d864e97a6f9b0f2d30fa533fbaa0b9155ad1a8495"} Jul 24 15:03:28.594269 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.594264 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r" Jul 24 15:03:28.595685 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.595661 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kx664" event={"ID":"5a357ffd-7231-4fb2-8fec-3b050c5a64bc","Type":"ContainerStarted","Data":"aed3f51d226e2c649320dc0e76ac197d6cfb2e030bc37d68cd4895e78b5dfde6"} Jul 24 15:03:28.595788 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.595720 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kx664" Jul 24 15:03:28.628626 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.628580 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r" podStartSLOduration=1.586087368 podStartE2EDuration="3.628563925s" totalImagesPullingTime="2.042476557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:03:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:03:28.626712757 +0000 UTC m=+440.857189441" watchObservedRunningTime="2026-07-24 15:03:28.628563925 +0000 UTC m=+440.859040608" Jul 24 15:03:28.667510 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.667460 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kx664" podStartSLOduration=3.143380716 podStartE2EDuration="5.667443743s" totalImagesPullingTime="2.524063027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:03:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:03:28.664837792 +0000 UTC m=+440.895314476" watchObservedRunningTime="2026-07-24 15:03:28.667443743 +0000 UTC m=+440.897920426" Jul 24 15:03:28.778692 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.778669 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 15:03:28.892985 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.892901 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-console-serving-cert\") pod \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " Jul 24 15:03:28.892985 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.892971 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-console-oauth-config\") pod \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " Jul 24 15:03:28.893183 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.893060 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnqmc\" (UniqueName: \"kubernetes.io/projected/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-kube-api-access-hnqmc\") pod \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " Jul 24 15:03:28.893183 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.893088 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-oauth-serving-cert\") pod \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " Jul 24 15:03:28.893183 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.893119 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-console-config\") pod \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " Jul 24 15:03:28.893183 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.893154 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-trusted-ca-bundle\") pod \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " Jul 24 15:03:28.893375 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.893200 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-service-ca\") pod \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\" (UID: \"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b\") " Jul 24 15:03:28.893725 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.893689 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b" (UID: "2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:03:28.893836 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.893743 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-service-ca" (OuterVolumeSpecName: "service-ca") pod "2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b" (UID: "2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:03:28.893836 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.893811 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b" (UID: "2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:03:28.893963 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.893608 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-console-config" (OuterVolumeSpecName: "console-config") pod "2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b" (UID: "2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:03:28.895916 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.895880 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b" (UID: "2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:03:28.896158 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.896132 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b" (UID: "2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:03:28.896211 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.896158 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-kube-api-access-hnqmc" (OuterVolumeSpecName: "kube-api-access-hnqmc") pod "2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b" (UID: "2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b"). InnerVolumeSpecName "kube-api-access-hnqmc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:03:28.994937 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.994905 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnqmc\" (UniqueName: \"kubernetes.io/projected/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-kube-api-access-hnqmc\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:28.994937 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.994932 2531 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-oauth-serving-cert\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:28.994937 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.994942 2531 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-console-config\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:28.995127 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.994951 2531 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-trusted-ca-bundle\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:28.995127 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.994960 2531 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-service-ca\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:28.995127 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.994968 2531 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-console-serving-cert\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:28.995127 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:28.994976 2531 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b-console-oauth-config\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:29.002388 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:29.002357 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:29.383160 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:29.383128 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:29.602413 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:29.602380 2531 generic.go:356] "Generic (PLEG): container finished" podID="2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b" containerID="7d5dc93bb81e4000ccb957598ac4902ff410bba13c07d4a941531a3e85341eac" exitCode=0 Jul 24 15:03:29.602553 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:29.602421 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b588b9796-8vjfp" Jul 24 15:03:29.602553 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:29.602465 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b588b9796-8vjfp" event={"ID":"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b","Type":"ContainerDied","Data":"7d5dc93bb81e4000ccb957598ac4902ff410bba13c07d4a941531a3e85341eac"} Jul 24 15:03:29.602553 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:29.602513 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b588b9796-8vjfp" event={"ID":"2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b","Type":"ContainerDied","Data":"5e1309738de87cd97bee8e9e16625784683c8d91ef7fc5cf0a0db9391654a683"} Jul 24 15:03:29.602553 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:29.602533 2531 scope.go:117] "RemoveContainer" containerID="7d5dc93bb81e4000ccb957598ac4902ff410bba13c07d4a941531a3e85341eac" Jul 24 15:03:29.612195 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:29.612177 2531 scope.go:117] "RemoveContainer" containerID="7d5dc93bb81e4000ccb957598ac4902ff410bba13c07d4a941531a3e85341eac" Jul 24 15:03:29.612459 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:03:29.612439 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d5dc93bb81e4000ccb957598ac4902ff410bba13c07d4a941531a3e85341eac\": container with ID starting with 7d5dc93bb81e4000ccb957598ac4902ff410bba13c07d4a941531a3e85341eac not found: ID does not exist" containerID="7d5dc93bb81e4000ccb957598ac4902ff410bba13c07d4a941531a3e85341eac" Jul 24 15:03:29.612505 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:29.612466 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d5dc93bb81e4000ccb957598ac4902ff410bba13c07d4a941531a3e85341eac"} err="failed to get container status \"7d5dc93bb81e4000ccb957598ac4902ff410bba13c07d4a941531a3e85341eac\": rpc error: code = NotFound desc = could not find container \"7d5dc93bb81e4000ccb957598ac4902ff410bba13c07d4a941531a3e85341eac\": container with ID starting with 7d5dc93bb81e4000ccb957598ac4902ff410bba13c07d4a941531a3e85341eac not found: ID does not exist" Jul 24 15:03:29.625571 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:29.625549 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b588b9796-8vjfp"] Jul 24 15:03:29.631343 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:29.631323 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b588b9796-8vjfp"] Jul 24 15:03:30.277755 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:30.277714 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b" path="/var/lib/kubelet/pods/2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b/volumes" Jul 24 15:03:36.763190 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:36.763155 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:36.870798 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:36.870760 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:37.588816 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:37.588788 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" Jul 24 15:03:37.589020 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:37.588994 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-s5kck" Jul 24 15:03:39.390377 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.390321 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:39.514486 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.514450 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:39.605459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.605434 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r" Jul 24 15:03:39.605602 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.605503 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kx664" Jul 24 15:03:39.632940 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.632913 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx"] Jul 24 15:03:39.633097 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.633077 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" podUID="58e333e1-d447-4c42-9c8e-158ddd899b06" containerName="manager" containerID="cri-o://3cb889eb7243e18f6f41066054b5d831a381cc869a4e0bda07396a7457fd5ab5" gracePeriod=2 Jul 24 15:03:39.643431 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.643367 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx"] Jul 24 15:03:39.654701 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.654673 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r"] Jul 24 15:03:39.656731 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:03:39.656701 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6wnxx\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="58e333e1-d447-4c42-9c8e-158ddd899b06" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" Jul 24 15:03:39.661794 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.661763 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r"] Jul 24 15:03:39.667141 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.667115 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln"] Jul 24 15:03:39.668377 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.667675 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e1ea8a8-9a24-48b0-910b-d43f2e8c951b" containerName="manager" Jul 24 15:03:39.668377 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.667698 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e1ea8a8-9a24-48b0-910b-d43f2e8c951b" containerName="manager" Jul 24 15:03:39.668377 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.667743 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58e333e1-d447-4c42-9c8e-158ddd899b06" containerName="manager" Jul 24 15:03:39.668377 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.667752 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="58e333e1-d447-4c42-9c8e-158ddd899b06" containerName="manager" Jul 24 15:03:39.668377 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.667769 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b" containerName="console" Jul 24 15:03:39.668377 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.667778 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b" containerName="console" Jul 24 15:03:39.668377 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.667903 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f6a3c4f-a702-4e35-9ce4-b0abbd0dce0b" containerName="console" Jul 24 15:03:39.668377 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.667917 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="58e333e1-d447-4c42-9c8e-158ddd899b06" containerName="manager" Jul 24 15:03:39.668377 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.667926 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e1ea8a8-9a24-48b0-910b-d43f2e8c951b" containerName="manager" Jul 24 15:03:39.671351 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.671318 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r" podUID="7e1ea8a8-9a24-48b0-910b-d43f2e8c951b" containerName="manager" containerID="cri-o://5556922f1174f17080cdf16d864e97a6f9b0f2d30fa533fbaa0b9155ad1a8495" gracePeriod=2 Jul 24 15:03:39.671600 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.671579 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln" Jul 24 15:03:39.673529 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:03:39.673495 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pf47r\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="7e1ea8a8-9a24-48b0-910b-d43f2e8c951b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r" Jul 24 15:03:39.675621 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:03:39.675597 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6wnxx\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="58e333e1-d447-4c42-9c8e-158ddd899b06" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" Jul 24 15:03:39.689635 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.689609 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln"] Jul 24 15:03:39.697365 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.697342 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbbpv"] Jul 24 15:03:39.701572 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.701551 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbbpv" Jul 24 15:03:39.704105 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:03:39.704074 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6wnxx\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="58e333e1-d447-4c42-9c8e-158ddd899b06" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" Jul 24 15:03:39.706189 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:03:39.706162 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pf47r\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="7e1ea8a8-9a24-48b0-910b-d43f2e8c951b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r" Jul 24 15:03:39.722661 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.722639 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbbpv"] Jul 24 15:03:39.746029 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:03:39.746003 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6wnxx\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="58e333e1-d447-4c42-9c8e-158ddd899b06" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" Jul 24 15:03:39.752193 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:03:39.752168 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pf47r\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="7e1ea8a8-9a24-48b0-910b-d43f2e8c951b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r" Jul 24 15:03:39.789683 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.789596 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9mdq\" (UniqueName: \"kubernetes.io/projected/094e5e26-87ea-4c9a-a49d-b44b9368b2bd-kube-api-access-k9mdq\") pod \"limitador-operator-controller-manager-859d7b78bd-sbbpv\" (UID: \"094e5e26-87ea-4c9a-a49d-b44b9368b2bd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbbpv" Jul 24 15:03:39.789750 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.789695 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca1560fa-caf0-4ca7-96eb-5038ea68450a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4ghln\" (UID: \"ca1560fa-caf0-4ca7-96eb-5038ea68450a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln" Jul 24 15:03:39.789750 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.789720 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vw6c\" (UniqueName: \"kubernetes.io/projected/ca1560fa-caf0-4ca7-96eb-5038ea68450a-kube-api-access-8vw6c\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4ghln\" (UID: \"ca1560fa-caf0-4ca7-96eb-5038ea68450a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln" Jul 24 15:03:39.891191 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.891160 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca1560fa-caf0-4ca7-96eb-5038ea68450a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4ghln\" (UID: \"ca1560fa-caf0-4ca7-96eb-5038ea68450a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln" Jul 24 15:03:39.891360 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.891194 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vw6c\" (UniqueName: \"kubernetes.io/projected/ca1560fa-caf0-4ca7-96eb-5038ea68450a-kube-api-access-8vw6c\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4ghln\" (UID: \"ca1560fa-caf0-4ca7-96eb-5038ea68450a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln" Jul 24 15:03:39.891360 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.891239 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9mdq\" (UniqueName: \"kubernetes.io/projected/094e5e26-87ea-4c9a-a49d-b44b9368b2bd-kube-api-access-k9mdq\") pod \"limitador-operator-controller-manager-859d7b78bd-sbbpv\" (UID: \"094e5e26-87ea-4c9a-a49d-b44b9368b2bd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbbpv" Jul 24 15:03:39.891629 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.891587 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca1560fa-caf0-4ca7-96eb-5038ea68450a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4ghln\" (UID: \"ca1560fa-caf0-4ca7-96eb-5038ea68450a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln" Jul 24 15:03:39.902752 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.902723 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9mdq\" (UniqueName: \"kubernetes.io/projected/094e5e26-87ea-4c9a-a49d-b44b9368b2bd-kube-api-access-k9mdq\") pod \"limitador-operator-controller-manager-859d7b78bd-sbbpv\" (UID: \"094e5e26-87ea-4c9a-a49d-b44b9368b2bd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbbpv" Jul 24 15:03:39.902990 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.902967 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vw6c\" (UniqueName: \"kubernetes.io/projected/ca1560fa-caf0-4ca7-96eb-5038ea68450a-kube-api-access-8vw6c\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4ghln\" (UID: \"ca1560fa-caf0-4ca7-96eb-5038ea68450a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln" Jul 24 15:03:39.928061 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.928039 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" Jul 24 15:03:39.930064 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:39.930044 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r" Jul 24 15:03:39.930362 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:03:39.930342 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pf47r\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="7e1ea8a8-9a24-48b0-910b-d43f2e8c951b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r" Jul 24 15:03:39.932421 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:03:39.932399 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6wnxx\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="58e333e1-d447-4c42-9c8e-158ddd899b06" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" Jul 24 15:03:39.935357 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:03:39.935337 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6wnxx\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="58e333e1-d447-4c42-9c8e-158ddd899b06" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" Jul 24 15:03:39.937297 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:03:39.937273 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pf47r\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="7e1ea8a8-9a24-48b0-910b-d43f2e8c951b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r" Jul 24 15:03:40.082439 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.082408 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln" Jul 24 15:03:40.089144 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.089127 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbbpv" Jul 24 15:03:40.093046 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.093026 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5qkm\" (UniqueName: \"kubernetes.io/projected/7e1ea8a8-9a24-48b0-910b-d43f2e8c951b-kube-api-access-h5qkm\") pod \"7e1ea8a8-9a24-48b0-910b-d43f2e8c951b\" (UID: \"7e1ea8a8-9a24-48b0-910b-d43f2e8c951b\") " Jul 24 15:03:40.093157 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.093071 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58e333e1-d447-4c42-9c8e-158ddd899b06-extensions-socket-volume\") pod \"58e333e1-d447-4c42-9c8e-158ddd899b06\" (UID: \"58e333e1-d447-4c42-9c8e-158ddd899b06\") " Jul 24 15:03:40.093157 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.093144 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhh2k\" (UniqueName: \"kubernetes.io/projected/58e333e1-d447-4c42-9c8e-158ddd899b06-kube-api-access-xhh2k\") pod \"58e333e1-d447-4c42-9c8e-158ddd899b06\" (UID: \"58e333e1-d447-4c42-9c8e-158ddd899b06\") " Jul 24 15:03:40.093571 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.093540 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58e333e1-d447-4c42-9c8e-158ddd899b06-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "58e333e1-d447-4c42-9c8e-158ddd899b06" (UID: "58e333e1-d447-4c42-9c8e-158ddd899b06"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:03:40.095458 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.095428 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58e333e1-d447-4c42-9c8e-158ddd899b06-kube-api-access-xhh2k" (OuterVolumeSpecName: "kube-api-access-xhh2k") pod "58e333e1-d447-4c42-9c8e-158ddd899b06" (UID: "58e333e1-d447-4c42-9c8e-158ddd899b06"). InnerVolumeSpecName "kube-api-access-xhh2k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:03:40.095527 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.095461 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e1ea8a8-9a24-48b0-910b-d43f2e8c951b-kube-api-access-h5qkm" (OuterVolumeSpecName: "kube-api-access-h5qkm") pod "7e1ea8a8-9a24-48b0-910b-d43f2e8c951b" (UID: "7e1ea8a8-9a24-48b0-910b-d43f2e8c951b"). InnerVolumeSpecName "kube-api-access-h5qkm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:03:40.194644 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.194611 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhh2k\" (UniqueName: \"kubernetes.io/projected/58e333e1-d447-4c42-9c8e-158ddd899b06-kube-api-access-xhh2k\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:40.194773 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.194649 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5qkm\" (UniqueName: \"kubernetes.io/projected/7e1ea8a8-9a24-48b0-910b-d43f2e8c951b-kube-api-access-h5qkm\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:40.194773 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.194666 2531 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58e333e1-d447-4c42-9c8e-158ddd899b06-extensions-socket-volume\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:03:40.252324 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.252299 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln"] Jul 24 15:03:40.255064 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:03:40.255030 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca1560fa_caf0_4ca7_96eb_5038ea68450a.slice/crio-f546af23c116af8e1378c9b076d953e86a9309c363cf81b239935cec88502417 WatchSource:0}: Error finding container f546af23c116af8e1378c9b076d953e86a9309c363cf81b239935cec88502417: Status 404 returned error can't find the container with id f546af23c116af8e1378c9b076d953e86a9309c363cf81b239935cec88502417 Jul 24 15:03:40.258895 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.258853 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbbpv"] Jul 24 15:03:40.260061 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:03:40.260041 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod094e5e26_87ea_4c9a_a49d_b44b9368b2bd.slice/crio-5a388988c881a48ca21232d71495534936ae9d70bca3a023e7a22721ff7b4955 WatchSource:0}: Error finding container 5a388988c881a48ca21232d71495534936ae9d70bca3a023e7a22721ff7b4955: Status 404 returned error can't find the container with id 5a388988c881a48ca21232d71495534936ae9d70bca3a023e7a22721ff7b4955 Jul 24 15:03:40.277840 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.277814 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58e333e1-d447-4c42-9c8e-158ddd899b06" path="/var/lib/kubelet/pods/58e333e1-d447-4c42-9c8e-158ddd899b06/volumes" Jul 24 15:03:40.278159 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.278146 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e1ea8a8-9a24-48b0-910b-d43f2e8c951b" path="/var/lib/kubelet/pods/7e1ea8a8-9a24-48b0-910b-d43f2e8c951b/volumes" Jul 24 15:03:40.679678 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.679641 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln" event={"ID":"ca1560fa-caf0-4ca7-96eb-5038ea68450a","Type":"ContainerStarted","Data":"c7535403e44c3df8d6e81def93414ec3de93a793ec9a13f818b5f7bb409b4c57"} Jul 24 15:03:40.680070 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.679705 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln" Jul 24 15:03:40.680070 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.679730 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln" event={"ID":"ca1560fa-caf0-4ca7-96eb-5038ea68450a","Type":"ContainerStarted","Data":"f546af23c116af8e1378c9b076d953e86a9309c363cf81b239935cec88502417"} Jul 24 15:03:40.681006 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.680981 2531 generic.go:356] "Generic (PLEG): container finished" podID="7e1ea8a8-9a24-48b0-910b-d43f2e8c951b" containerID="5556922f1174f17080cdf16d864e97a6f9b0f2d30fa533fbaa0b9155ad1a8495" exitCode=0 Jul 24 15:03:40.681066 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.681038 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pf47r" Jul 24 15:03:40.681109 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.681072 2531 scope.go:117] "RemoveContainer" containerID="5556922f1174f17080cdf16d864e97a6f9b0f2d30fa533fbaa0b9155ad1a8495" Jul 24 15:03:40.683433 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.683402 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbbpv" event={"ID":"094e5e26-87ea-4c9a-a49d-b44b9368b2bd","Type":"ContainerStarted","Data":"f1a50041965185b56693f6e468fbbe51042ced35e389e313fd51b5f5f8aafd47"} Jul 24 15:03:40.683567 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.683459 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbbpv" event={"ID":"094e5e26-87ea-4c9a-a49d-b44b9368b2bd","Type":"ContainerStarted","Data":"5a388988c881a48ca21232d71495534936ae9d70bca3a023e7a22721ff7b4955"} Jul 24 15:03:40.684056 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.684012 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbbpv" Jul 24 15:03:40.685527 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.685502 2531 generic.go:356] "Generic (PLEG): container finished" podID="58e333e1-d447-4c42-9c8e-158ddd899b06" containerID="3cb889eb7243e18f6f41066054b5d831a381cc869a4e0bda07396a7457fd5ab5" exitCode=0 Jul 24 15:03:40.685643 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.685572 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6wnxx" Jul 24 15:03:40.694059 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.694040 2531 scope.go:117] "RemoveContainer" containerID="5556922f1174f17080cdf16d864e97a6f9b0f2d30fa533fbaa0b9155ad1a8495" Jul 24 15:03:40.694335 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:03:40.694320 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5556922f1174f17080cdf16d864e97a6f9b0f2d30fa533fbaa0b9155ad1a8495\": container with ID starting with 5556922f1174f17080cdf16d864e97a6f9b0f2d30fa533fbaa0b9155ad1a8495 not found: ID does not exist" containerID="5556922f1174f17080cdf16d864e97a6f9b0f2d30fa533fbaa0b9155ad1a8495" Jul 24 15:03:40.694383 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.694348 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5556922f1174f17080cdf16d864e97a6f9b0f2d30fa533fbaa0b9155ad1a8495"} err="failed to get container status \"5556922f1174f17080cdf16d864e97a6f9b0f2d30fa533fbaa0b9155ad1a8495\": rpc error: code = NotFound desc = could not find container \"5556922f1174f17080cdf16d864e97a6f9b0f2d30fa533fbaa0b9155ad1a8495\": container with ID starting with 5556922f1174f17080cdf16d864e97a6f9b0f2d30fa533fbaa0b9155ad1a8495 not found: ID does not exist" Jul 24 15:03:40.694383 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.694365 2531 scope.go:117] "RemoveContainer" containerID="3cb889eb7243e18f6f41066054b5d831a381cc869a4e0bda07396a7457fd5ab5" Jul 24 15:03:40.702799 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.702783 2531 scope.go:117] "RemoveContainer" containerID="3cb889eb7243e18f6f41066054b5d831a381cc869a4e0bda07396a7457fd5ab5" Jul 24 15:03:40.703087 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:03:40.703069 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cb889eb7243e18f6f41066054b5d831a381cc869a4e0bda07396a7457fd5ab5\": container with ID starting with 3cb889eb7243e18f6f41066054b5d831a381cc869a4e0bda07396a7457fd5ab5 not found: ID does not exist" containerID="3cb889eb7243e18f6f41066054b5d831a381cc869a4e0bda07396a7457fd5ab5" Jul 24 15:03:40.703136 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.703093 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cb889eb7243e18f6f41066054b5d831a381cc869a4e0bda07396a7457fd5ab5"} err="failed to get container status \"3cb889eb7243e18f6f41066054b5d831a381cc869a4e0bda07396a7457fd5ab5\": rpc error: code = NotFound desc = could not find container \"3cb889eb7243e18f6f41066054b5d831a381cc869a4e0bda07396a7457fd5ab5\": container with ID starting with 3cb889eb7243e18f6f41066054b5d831a381cc869a4e0bda07396a7457fd5ab5 not found: ID does not exist" Jul 24 15:03:40.722976 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.722937 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln" podStartSLOduration=1.722923735 podStartE2EDuration="1.722923735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:03:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:03:40.722248068 +0000 UTC m=+452.952724782" watchObservedRunningTime="2026-07-24 15:03:40.722923735 +0000 UTC m=+452.953400418" Jul 24 15:03:40.792499 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:40.792450 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbbpv" podStartSLOduration=1.792435923 podStartE2EDuration="1.792435923s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:03:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:03:40.789205234 +0000 UTC m=+453.019681918" watchObservedRunningTime="2026-07-24 15:03:40.792435923 +0000 UTC m=+453.022912607" Jul 24 15:03:43.984067 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:43.984022 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:44.099622 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:44.099586 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:47.920317 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:47.920274 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:48.072703 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:48.072638 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:51.696480 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:51.696445 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln" Jul 24 15:03:52.701430 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:52.701396 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sbbpv" Jul 24 15:03:56.520995 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:56.520954 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:56.707928 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:56.707889 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:58.963553 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:58.963519 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:03:59.051400 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:03:59.051370 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:03.743810 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:03.743772 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:03.846583 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:03.846550 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:07.894641 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:07.894604 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:07.994781 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:07.994739 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:09.074784 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:09.074753 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln"] Jul 24 15:04:09.075193 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:09.074960 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln" podUID="ca1560fa-caf0-4ca7-96eb-5038ea68450a" containerName="manager" containerID="cri-o://c7535403e44c3df8d6e81def93414ec3de93a793ec9a13f818b5f7bb409b4c57" gracePeriod=10 Jul 24 15:04:09.330590 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:09.330525 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln" Jul 24 15:04:09.446265 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:09.446231 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vw6c\" (UniqueName: \"kubernetes.io/projected/ca1560fa-caf0-4ca7-96eb-5038ea68450a-kube-api-access-8vw6c\") pod \"ca1560fa-caf0-4ca7-96eb-5038ea68450a\" (UID: \"ca1560fa-caf0-4ca7-96eb-5038ea68450a\") " Jul 24 15:04:09.446440 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:09.446324 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca1560fa-caf0-4ca7-96eb-5038ea68450a-extensions-socket-volume\") pod \"ca1560fa-caf0-4ca7-96eb-5038ea68450a\" (UID: \"ca1560fa-caf0-4ca7-96eb-5038ea68450a\") " Jul 24 15:04:09.446706 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:09.446685 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca1560fa-caf0-4ca7-96eb-5038ea68450a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ca1560fa-caf0-4ca7-96eb-5038ea68450a" (UID: "ca1560fa-caf0-4ca7-96eb-5038ea68450a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:04:09.448466 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:09.448446 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca1560fa-caf0-4ca7-96eb-5038ea68450a-kube-api-access-8vw6c" (OuterVolumeSpecName: "kube-api-access-8vw6c") pod "ca1560fa-caf0-4ca7-96eb-5038ea68450a" (UID: "ca1560fa-caf0-4ca7-96eb-5038ea68450a"). InnerVolumeSpecName "kube-api-access-8vw6c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:04:09.547940 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:09.547906 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vw6c\" (UniqueName: \"kubernetes.io/projected/ca1560fa-caf0-4ca7-96eb-5038ea68450a-kube-api-access-8vw6c\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:04:09.547940 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:09.547933 2531 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ca1560fa-caf0-4ca7-96eb-5038ea68450a-extensions-socket-volume\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:04:09.859048 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:09.859009 2531 generic.go:356] "Generic (PLEG): container finished" podID="ca1560fa-caf0-4ca7-96eb-5038ea68450a" containerID="c7535403e44c3df8d6e81def93414ec3de93a793ec9a13f818b5f7bb409b4c57" exitCode=0 Jul 24 15:04:09.859260 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:09.859059 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln" Jul 24 15:04:09.859260 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:09.859093 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln" event={"ID":"ca1560fa-caf0-4ca7-96eb-5038ea68450a","Type":"ContainerDied","Data":"c7535403e44c3df8d6e81def93414ec3de93a793ec9a13f818b5f7bb409b4c57"} Jul 24 15:04:09.859260 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:09.859158 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln" event={"ID":"ca1560fa-caf0-4ca7-96eb-5038ea68450a","Type":"ContainerDied","Data":"f546af23c116af8e1378c9b076d953e86a9309c363cf81b239935cec88502417"} Jul 24 15:04:09.859260 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:09.859194 2531 scope.go:117] "RemoveContainer" containerID="c7535403e44c3df8d6e81def93414ec3de93a793ec9a13f818b5f7bb409b4c57" Jul 24 15:04:09.869029 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:09.869014 2531 scope.go:117] "RemoveContainer" containerID="c7535403e44c3df8d6e81def93414ec3de93a793ec9a13f818b5f7bb409b4c57" Jul 24 15:04:09.869282 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:04:09.869261 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7535403e44c3df8d6e81def93414ec3de93a793ec9a13f818b5f7bb409b4c57\": container with ID starting with c7535403e44c3df8d6e81def93414ec3de93a793ec9a13f818b5f7bb409b4c57 not found: ID does not exist" containerID="c7535403e44c3df8d6e81def93414ec3de93a793ec9a13f818b5f7bb409b4c57" Jul 24 15:04:09.869323 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:09.869290 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7535403e44c3df8d6e81def93414ec3de93a793ec9a13f818b5f7bb409b4c57"} err="failed to get container status \"c7535403e44c3df8d6e81def93414ec3de93a793ec9a13f818b5f7bb409b4c57\": rpc error: code = NotFound desc = could not find container \"c7535403e44c3df8d6e81def93414ec3de93a793ec9a13f818b5f7bb409b4c57\": container with ID starting with c7535403e44c3df8d6e81def93414ec3de93a793ec9a13f818b5f7bb409b4c57 not found: ID does not exist" Jul 24 15:04:09.887785 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:09.887764 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln"] Jul 24 15:04:09.894824 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:09.894806 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4ghln"] Jul 24 15:04:10.278275 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:10.278241 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca1560fa-caf0-4ca7-96eb-5038ea68450a" path="/var/lib/kubelet/pods/ca1560fa-caf0-4ca7-96eb-5038ea68450a/volumes" Jul 24 15:04:16.908702 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:16.908652 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:17.001084 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:17.001052 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:19.380423 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:19.380386 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:19.486085 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:19.486039 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:24.682914 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:24.682860 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:24.783171 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:24.783132 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:25.308330 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.308300 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:25.408729 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.408695 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:25.524133 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.524107 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst"] Jul 24 15:04:25.524508 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.524494 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca1560fa-caf0-4ca7-96eb-5038ea68450a" containerName="manager" Jul 24 15:04:25.524551 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.524509 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca1560fa-caf0-4ca7-96eb-5038ea68450a" containerName="manager" Jul 24 15:04:25.524596 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.524587 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca1560fa-caf0-4ca7-96eb-5038ea68450a" containerName="manager" Jul 24 15:04:25.527905 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.527860 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.530568 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.530546 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-7vgk7\"" Jul 24 15:04:25.530568 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.530555 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 15:04:25.546286 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.546262 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst"] Jul 24 15:04:25.555573 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.555545 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:25.686072 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.686045 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.686408 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.686075 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.686408 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.686102 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62gml\" (UniqueName: \"kubernetes.io/projected/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-kube-api-access-62gml\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.686408 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.686180 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.686408 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.686202 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.686408 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.686232 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.686408 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.686251 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.686408 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.686308 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.686408 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.686325 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.754436 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.754401 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:25.787346 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.787321 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.787445 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.787351 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.787445 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.787412 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.787445 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.787431 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.787598 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.787448 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.787598 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.787472 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.787598 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.787511 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62gml\" (UniqueName: \"kubernetes.io/projected/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-kube-api-access-62gml\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.787598 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.787585 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.787794 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.787607 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.787858 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.787810 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.788044 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.788022 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.788123 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.788107 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.788179 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.788161 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.788333 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.788317 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.790092 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.790073 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.790188 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.790168 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.796075 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.796054 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.796280 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.796263 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62gml\" (UniqueName: \"kubernetes.io/projected/0e873231-3e07-4d1d-9d0d-bf7cb190d9c3-kube-api-access-62gml\") pod \"maas-default-gateway-openshift-default-78bb78ff78-gzdst\" (UID: \"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.838248 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.838206 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:25.978751 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:25.978692 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst"] Jul 24 15:04:25.981233 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:04:25.981206 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e873231_3e07_4d1d_9d0d_bf7cb190d9c3.slice/crio-f22959e801355fd8ad5127bd6897a63905f8a7779525d0a1bc33e29e03a3da50 WatchSource:0}: Error finding container f22959e801355fd8ad5127bd6897a63905f8a7779525d0a1bc33e29e03a3da50: Status 404 returned error can't find the container with id f22959e801355fd8ad5127bd6897a63905f8a7779525d0a1bc33e29e03a3da50 Jul 24 15:04:26.960883 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:26.960826 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" event={"ID":"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3","Type":"ContainerStarted","Data":"f22959e801355fd8ad5127bd6897a63905f8a7779525d0a1bc33e29e03a3da50"} Jul 24 15:04:29.030380 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:29.030347 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 24 15:04:29.030626 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:29.030418 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 24 15:04:29.030626 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:29.030444 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 24 15:04:29.980302 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:29.980262 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" event={"ID":"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3","Type":"ContainerStarted","Data":"4c6465b05df96203d5bb98a94fe3e202761c0bf73d11bbbb20aec426b8073eb1"} Jul 24 15:04:30.010289 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:30.010238 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" podStartSLOduration=1.963360692 podStartE2EDuration="5.010223386s" totalImagesPullingTime="3.046862694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:04:30.009456431 +0000 UTC m=+502.239933112" watchObservedRunningTime="2026-07-24 15:04:30.010223386 +0000 UTC m=+502.240700069" Jul 24 15:04:30.839015 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:30.838987 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:30.843927 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:30.843903 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:30.986021 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:30.985994 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:30.986975 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:30.986957 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:04:31.141693 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.141618 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:31.142779 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.142758 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mzp7f"] Jul 24 15:04:31.146239 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.146224 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mzp7f" Jul 24 15:04:31.149115 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.149091 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 24 15:04:31.149230 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.149126 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-nd7qj\"" Jul 24 15:04:31.165960 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.165928 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mzp7f"] Jul 24 15:04:31.237017 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.236986 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4db2e004-6a36-451f-8325-d88e81f666db-config-file\") pod \"limitador-limitador-64c94fd64f-mzp7f\" (UID: \"4db2e004-6a36-451f-8325-d88e81f666db\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mzp7f" Jul 24 15:04:31.237197 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.237132 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzcdt\" (UniqueName: \"kubernetes.io/projected/4db2e004-6a36-451f-8325-d88e81f666db-kube-api-access-zzcdt\") pod \"limitador-limitador-64c94fd64f-mzp7f\" (UID: \"4db2e004-6a36-451f-8325-d88e81f666db\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mzp7f" Jul 24 15:04:31.237197 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.237147 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mzp7f"] Jul 24 15:04:31.266336 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.266303 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:31.338329 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.338293 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzcdt\" (UniqueName: \"kubernetes.io/projected/4db2e004-6a36-451f-8325-d88e81f666db-kube-api-access-zzcdt\") pod \"limitador-limitador-64c94fd64f-mzp7f\" (UID: \"4db2e004-6a36-451f-8325-d88e81f666db\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mzp7f" Jul 24 15:04:31.338485 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.338408 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4db2e004-6a36-451f-8325-d88e81f666db-config-file\") pod \"limitador-limitador-64c94fd64f-mzp7f\" (UID: \"4db2e004-6a36-451f-8325-d88e81f666db\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mzp7f" Jul 24 15:04:31.339042 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.339020 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4db2e004-6a36-451f-8325-d88e81f666db-config-file\") pod \"limitador-limitador-64c94fd64f-mzp7f\" (UID: \"4db2e004-6a36-451f-8325-d88e81f666db\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mzp7f" Jul 24 15:04:31.346933 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.346907 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzcdt\" (UniqueName: \"kubernetes.io/projected/4db2e004-6a36-451f-8325-d88e81f666db-kube-api-access-zzcdt\") pod \"limitador-limitador-64c94fd64f-mzp7f\" (UID: \"4db2e004-6a36-451f-8325-d88e81f666db\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-mzp7f" Jul 24 15:04:31.365194 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.365149 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:31.456331 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.456297 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mzp7f" Jul 24 15:04:31.489540 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.489499 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:04:31.493125 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.493104 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-ljfxb" Jul 24 15:04:31.505620 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.505594 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:04:31.538479 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.538452 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:04:31.600754 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.600726 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mzp7f"] Jul 24 15:04:31.602628 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:04:31.602600 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4db2e004_6a36_451f_8325_d88e81f666db.slice/crio-c5a9322c16a2fa918779be5953a6637246c7157a3885562ba41d4d7cc5b5a70b WatchSource:0}: Error finding container c5a9322c16a2fa918779be5953a6637246c7157a3885562ba41d4d7cc5b5a70b: Status 404 returned error can't find the container with id c5a9322c16a2fa918779be5953a6637246c7157a3885562ba41d4d7cc5b5a70b Jul 24 15:04:31.640527 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.640501 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqqt4\" (UniqueName: \"kubernetes.io/projected/34a9573a-009d-41e7-802f-4578118e3d02-kube-api-access-jqqt4\") pod \"limitador-limitador-54d6d988d5-ljfxb\" (UID: \"34a9573a-009d-41e7-802f-4578118e3d02\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ljfxb" Jul 24 15:04:31.640652 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.640567 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/34a9573a-009d-41e7-802f-4578118e3d02-config-file\") pod \"limitador-limitador-54d6d988d5-ljfxb\" (UID: \"34a9573a-009d-41e7-802f-4578118e3d02\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ljfxb" Jul 24 15:04:31.741677 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.741603 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqqt4\" (UniqueName: \"kubernetes.io/projected/34a9573a-009d-41e7-802f-4578118e3d02-kube-api-access-jqqt4\") pod \"limitador-limitador-54d6d988d5-ljfxb\" (UID: \"34a9573a-009d-41e7-802f-4578118e3d02\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ljfxb" Jul 24 15:04:31.741677 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.741659 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/34a9573a-009d-41e7-802f-4578118e3d02-config-file\") pod \"limitador-limitador-54d6d988d5-ljfxb\" (UID: \"34a9573a-009d-41e7-802f-4578118e3d02\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ljfxb" Jul 24 15:04:31.742234 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.742215 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/34a9573a-009d-41e7-802f-4578118e3d02-config-file\") pod \"limitador-limitador-54d6d988d5-ljfxb\" (UID: \"34a9573a-009d-41e7-802f-4578118e3d02\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ljfxb" Jul 24 15:04:31.751193 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.751165 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqqt4\" (UniqueName: \"kubernetes.io/projected/34a9573a-009d-41e7-802f-4578118e3d02-kube-api-access-jqqt4\") pod \"limitador-limitador-54d6d988d5-ljfxb\" (UID: \"34a9573a-009d-41e7-802f-4578118e3d02\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-ljfxb" Jul 24 15:04:31.805954 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.805935 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-ljfxb" Jul 24 15:04:31.949812 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.949765 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:04:31.953550 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:04:31.953523 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34a9573a_009d_41e7_802f_4578118e3d02.slice/crio-9ca4ddbfe146a8f465cd46dea2463379f9c5403290a2b85a79dec4d9b12ead9a WatchSource:0}: Error finding container 9ca4ddbfe146a8f465cd46dea2463379f9c5403290a2b85a79dec4d9b12ead9a: Status 404 returned error can't find the container with id 9ca4ddbfe146a8f465cd46dea2463379f9c5403290a2b85a79dec4d9b12ead9a Jul 24 15:04:31.953550 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.953539 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-qnjml"] Jul 24 15:04:31.958222 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.958205 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-qnjml" Jul 24 15:04:31.960691 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.960669 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hdm24\"" Jul 24 15:04:31.962726 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.962706 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-qnjml"] Jul 24 15:04:31.992857 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.992802 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mzp7f" event={"ID":"4db2e004-6a36-451f-8325-d88e81f666db","Type":"ContainerStarted","Data":"c5a9322c16a2fa918779be5953a6637246c7157a3885562ba41d4d7cc5b5a70b"} Jul 24 15:04:31.994289 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.994269 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-ljfxb" event={"ID":"34a9573a-009d-41e7-802f-4578118e3d02","Type":"ContainerStarted","Data":"9ca4ddbfe146a8f465cd46dea2463379f9c5403290a2b85a79dec4d9b12ead9a"} Jul 24 15:04:31.995679 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:31.995655 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:32.044791 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:32.044767 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fptvz\" (UniqueName: \"kubernetes.io/projected/535151df-d260-4823-be00-0f89ddb2a9ad-kube-api-access-fptvz\") pod \"authorino-58cd955675-qnjml\" (UID: \"535151df-d260-4823-be00-0f89ddb2a9ad\") " pod="kuadrant-system/authorino-58cd955675-qnjml" Jul 24 15:04:32.071993 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:32.071971 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:32.145518 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:32.145489 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fptvz\" (UniqueName: \"kubernetes.io/projected/535151df-d260-4823-be00-0f89ddb2a9ad-kube-api-access-fptvz\") pod \"authorino-58cd955675-qnjml\" (UID: \"535151df-d260-4823-be00-0f89ddb2a9ad\") " pod="kuadrant-system/authorino-58cd955675-qnjml" Jul 24 15:04:32.157826 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:32.157788 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fptvz\" (UniqueName: \"kubernetes.io/projected/535151df-d260-4823-be00-0f89ddb2a9ad-kube-api-access-fptvz\") pod \"authorino-58cd955675-qnjml\" (UID: \"535151df-d260-4823-be00-0f89ddb2a9ad\") " pod="kuadrant-system/authorino-58cd955675-qnjml" Jul 24 15:04:32.269331 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:32.269266 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-qnjml" Jul 24 15:04:32.446726 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:32.446701 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-qnjml"] Jul 24 15:04:32.450285 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:04:32.450258 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod535151df_d260_4823_be00_0f89ddb2a9ad.slice/crio-b5c8077261cb9038240058797ce5b144593d55bd1367dc83aa65b6aec37aa572 WatchSource:0}: Error finding container b5c8077261cb9038240058797ce5b144593d55bd1367dc83aa65b6aec37aa572: Status 404 returned error can't find the container with id b5c8077261cb9038240058797ce5b144593d55bd1367dc83aa65b6aec37aa572 Jul 24 15:04:33.004884 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:33.004832 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-qnjml" event={"ID":"535151df-d260-4823-be00-0f89ddb2a9ad","Type":"ContainerStarted","Data":"b5c8077261cb9038240058797ce5b144593d55bd1367dc83aa65b6aec37aa572"} Jul 24 15:04:36.204349 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:36.204309 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-qnjml"] Jul 24 15:04:37.035748 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:37.035713 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-ljfxb" event={"ID":"34a9573a-009d-41e7-802f-4578118e3d02","Type":"ContainerStarted","Data":"528aead5980cd420dae4871b6b5f8b36e0cda48a721564900cd478c8a48b6782"} Jul 24 15:04:37.035958 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:37.035777 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-ljfxb" Jul 24 15:04:37.037315 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:37.037284 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-qnjml" podUID="535151df-d260-4823-be00-0f89ddb2a9ad" containerName="authorino" containerID="cri-o://2601b3bb2081ca30434a06a4ddf9b810d10cbccd4420da0bceddbb5272a287e8" gracePeriod=30 Jul 24 15:04:37.037315 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:37.037309 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-qnjml" event={"ID":"535151df-d260-4823-be00-0f89ddb2a9ad","Type":"ContainerStarted","Data":"2601b3bb2081ca30434a06a4ddf9b810d10cbccd4420da0bceddbb5272a287e8"} Jul 24 15:04:37.038877 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:37.038823 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mzp7f" event={"ID":"4db2e004-6a36-451f-8325-d88e81f666db","Type":"ContainerStarted","Data":"daea38e3658f28ecd49560bd6228328827bfb57792eed2aab660610fd19959ac"} Jul 24 15:04:37.038994 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:37.038980 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-mzp7f" Jul 24 15:04:37.052102 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:37.052067 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-ljfxb" podStartSLOduration=1.430277362 podStartE2EDuration="6.052054046s" totalImagesPullingTime="4.621776684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:04:37.051283688 +0000 UTC m=+509.281760390" watchObservedRunningTime="2026-07-24 15:04:37.052054046 +0000 UTC m=+509.282530730" Jul 24 15:04:37.067961 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:37.067914 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-mzp7f" podStartSLOduration=1.086339137 podStartE2EDuration="6.06789952s" totalImagesPullingTime="4.981560383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:04:37.067434926 +0000 UTC m=+509.297911630" watchObservedRunningTime="2026-07-24 15:04:37.06789952 +0000 UTC m=+509.298376200" Jul 24 15:04:37.084073 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:37.084035 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-qnjml" podStartSLOduration=2.007918985 podStartE2EDuration="6.084024467s" totalImagesPullingTime="4.076105482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:04:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:04:37.082152531 +0000 UTC m=+509.312629215" watchObservedRunningTime="2026-07-24 15:04:37.084024467 +0000 UTC m=+509.314501150" Jul 24 15:04:37.283442 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:37.283421 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-qnjml" Jul 24 15:04:37.399436 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:37.399357 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fptvz\" (UniqueName: \"kubernetes.io/projected/535151df-d260-4823-be00-0f89ddb2a9ad-kube-api-access-fptvz\") pod \"535151df-d260-4823-be00-0f89ddb2a9ad\" (UID: \"535151df-d260-4823-be00-0f89ddb2a9ad\") " Jul 24 15:04:37.401546 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:37.401521 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/535151df-d260-4823-be00-0f89ddb2a9ad-kube-api-access-fptvz" (OuterVolumeSpecName: "kube-api-access-fptvz") pod "535151df-d260-4823-be00-0f89ddb2a9ad" (UID: "535151df-d260-4823-be00-0f89ddb2a9ad"). InnerVolumeSpecName "kube-api-access-fptvz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:04:37.500421 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:37.500394 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fptvz\" (UniqueName: \"kubernetes.io/projected/535151df-d260-4823-be00-0f89ddb2a9ad-kube-api-access-fptvz\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:04:38.045611 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:38.045576 2531 generic.go:356] "Generic (PLEG): container finished" podID="535151df-d260-4823-be00-0f89ddb2a9ad" containerID="2601b3bb2081ca30434a06a4ddf9b810d10cbccd4420da0bceddbb5272a287e8" exitCode=0 Jul 24 15:04:38.045753 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:38.045621 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-qnjml" Jul 24 15:04:38.045753 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:38.045664 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-qnjml" event={"ID":"535151df-d260-4823-be00-0f89ddb2a9ad","Type":"ContainerDied","Data":"2601b3bb2081ca30434a06a4ddf9b810d10cbccd4420da0bceddbb5272a287e8"} Jul 24 15:04:38.045753 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:38.045704 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-qnjml" event={"ID":"535151df-d260-4823-be00-0f89ddb2a9ad","Type":"ContainerDied","Data":"b5c8077261cb9038240058797ce5b144593d55bd1367dc83aa65b6aec37aa572"} Jul 24 15:04:38.045753 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:38.045725 2531 scope.go:117] "RemoveContainer" containerID="2601b3bb2081ca30434a06a4ddf9b810d10cbccd4420da0bceddbb5272a287e8" Jul 24 15:04:38.054830 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:38.054815 2531 scope.go:117] "RemoveContainer" containerID="2601b3bb2081ca30434a06a4ddf9b810d10cbccd4420da0bceddbb5272a287e8" Jul 24 15:04:38.055119 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:04:38.055101 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2601b3bb2081ca30434a06a4ddf9b810d10cbccd4420da0bceddbb5272a287e8\": container with ID starting with 2601b3bb2081ca30434a06a4ddf9b810d10cbccd4420da0bceddbb5272a287e8 not found: ID does not exist" containerID="2601b3bb2081ca30434a06a4ddf9b810d10cbccd4420da0bceddbb5272a287e8" Jul 24 15:04:38.055176 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:38.055128 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2601b3bb2081ca30434a06a4ddf9b810d10cbccd4420da0bceddbb5272a287e8"} err="failed to get container status \"2601b3bb2081ca30434a06a4ddf9b810d10cbccd4420da0bceddbb5272a287e8\": rpc error: code = NotFound desc = could not find container \"2601b3bb2081ca30434a06a4ddf9b810d10cbccd4420da0bceddbb5272a287e8\": container with ID starting with 2601b3bb2081ca30434a06a4ddf9b810d10cbccd4420da0bceddbb5272a287e8 not found: ID does not exist" Jul 24 15:04:38.069358 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:38.069333 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-qnjml"] Jul 24 15:04:38.075117 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:38.075097 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-qnjml"] Jul 24 15:04:38.278385 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:38.278342 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="535151df-d260-4823-be00-0f89ddb2a9ad" path="/var/lib/kubelet/pods/535151df-d260-4823-be00-0f89ddb2a9ad/volumes" Jul 24 15:04:48.047403 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:48.047375 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-mzp7f" Jul 24 15:04:48.047784 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:48.047445 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-ljfxb" Jul 24 15:04:48.113280 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:48.113248 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mzp7f"] Jul 24 15:04:48.113438 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:48.113409 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-mzp7f" podUID="4db2e004-6a36-451f-8325-d88e81f666db" containerName="limitador" containerID="cri-o://daea38e3658f28ecd49560bd6228328827bfb57792eed2aab660610fd19959ac" gracePeriod=30 Jul 24 15:04:48.671372 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:48.671345 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mzp7f" Jul 24 15:04:48.691907 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:48.691829 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzcdt\" (UniqueName: \"kubernetes.io/projected/4db2e004-6a36-451f-8325-d88e81f666db-kube-api-access-zzcdt\") pod \"4db2e004-6a36-451f-8325-d88e81f666db\" (UID: \"4db2e004-6a36-451f-8325-d88e81f666db\") " Jul 24 15:04:48.692030 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:48.691957 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4db2e004-6a36-451f-8325-d88e81f666db-config-file\") pod \"4db2e004-6a36-451f-8325-d88e81f666db\" (UID: \"4db2e004-6a36-451f-8325-d88e81f666db\") " Jul 24 15:04:48.692324 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:48.692297 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4db2e004-6a36-451f-8325-d88e81f666db-config-file" (OuterVolumeSpecName: "config-file") pod "4db2e004-6a36-451f-8325-d88e81f666db" (UID: "4db2e004-6a36-451f-8325-d88e81f666db"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:04:48.694033 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:48.694010 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4db2e004-6a36-451f-8325-d88e81f666db-kube-api-access-zzcdt" (OuterVolumeSpecName: "kube-api-access-zzcdt") pod "4db2e004-6a36-451f-8325-d88e81f666db" (UID: "4db2e004-6a36-451f-8325-d88e81f666db"). InnerVolumeSpecName "kube-api-access-zzcdt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:04:48.793616 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:48.793589 2531 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4db2e004-6a36-451f-8325-d88e81f666db-config-file\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:04:48.793616 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:48.793612 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzcdt\" (UniqueName: \"kubernetes.io/projected/4db2e004-6a36-451f-8325-d88e81f666db-kube-api-access-zzcdt\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:04:49.107898 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:49.107793 2531 generic.go:356] "Generic (PLEG): container finished" podID="4db2e004-6a36-451f-8325-d88e81f666db" containerID="daea38e3658f28ecd49560bd6228328827bfb57792eed2aab660610fd19959ac" exitCode=0 Jul 24 15:04:49.107898 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:49.107833 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-mzp7f" Jul 24 15:04:49.107898 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:49.107890 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mzp7f" event={"ID":"4db2e004-6a36-451f-8325-d88e81f666db","Type":"ContainerDied","Data":"daea38e3658f28ecd49560bd6228328827bfb57792eed2aab660610fd19959ac"} Jul 24 15:04:49.108413 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:49.107940 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-mzp7f" event={"ID":"4db2e004-6a36-451f-8325-d88e81f666db","Type":"ContainerDied","Data":"c5a9322c16a2fa918779be5953a6637246c7157a3885562ba41d4d7cc5b5a70b"} Jul 24 15:04:49.108413 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:49.107962 2531 scope.go:117] "RemoveContainer" containerID="daea38e3658f28ecd49560bd6228328827bfb57792eed2aab660610fd19959ac" Jul 24 15:04:49.118188 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:49.117993 2531 scope.go:117] "RemoveContainer" containerID="daea38e3658f28ecd49560bd6228328827bfb57792eed2aab660610fd19959ac" Jul 24 15:04:49.118267 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:04:49.118251 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daea38e3658f28ecd49560bd6228328827bfb57792eed2aab660610fd19959ac\": container with ID starting with daea38e3658f28ecd49560bd6228328827bfb57792eed2aab660610fd19959ac not found: ID does not exist" containerID="daea38e3658f28ecd49560bd6228328827bfb57792eed2aab660610fd19959ac" Jul 24 15:04:49.118313 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:49.118274 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daea38e3658f28ecd49560bd6228328827bfb57792eed2aab660610fd19959ac"} err="failed to get container status \"daea38e3658f28ecd49560bd6228328827bfb57792eed2aab660610fd19959ac\": rpc error: code = NotFound desc = could not find container \"daea38e3658f28ecd49560bd6228328827bfb57792eed2aab660610fd19959ac\": container with ID starting with daea38e3658f28ecd49560bd6228328827bfb57792eed2aab660610fd19959ac not found: ID does not exist" Jul 24 15:04:49.131040 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:49.131020 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mzp7f"] Jul 24 15:04:49.139686 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:49.139667 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-mzp7f"] Jul 24 15:04:50.277668 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:50.277634 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4db2e004-6a36-451f-8325-d88e81f666db" path="/var/lib/kubelet/pods/4db2e004-6a36-451f-8325-d88e81f666db/volumes" Jul 24 15:04:52.439533 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:52.439495 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:52.558019 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:52.557986 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:52.968391 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:52.968350 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:04:57.503647 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:04:57.503615 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:05:02.631929 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:02.631797 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:05:07.070416 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:07.070361 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:05:07.277982 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:07.277949 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:05:07.377097 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:07.377021 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:05:14.264530 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.264494 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7dc5f4dd74-mwpfj"] Jul 24 15:05:14.264980 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.264965 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4db2e004-6a36-451f-8325-d88e81f666db" containerName="limitador" Jul 24 15:05:14.265030 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.264981 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="4db2e004-6a36-451f-8325-d88e81f666db" containerName="limitador" Jul 24 15:05:14.265030 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.265004 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="535151df-d260-4823-be00-0f89ddb2a9ad" containerName="authorino" Jul 24 15:05:14.265030 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.265013 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="535151df-d260-4823-be00-0f89ddb2a9ad" containerName="authorino" Jul 24 15:05:14.265134 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.265092 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="4db2e004-6a36-451f-8325-d88e81f666db" containerName="limitador" Jul 24 15:05:14.265134 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.265105 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="535151df-d260-4823-be00-0f89ddb2a9ad" containerName="authorino" Jul 24 15:05:14.268391 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.268358 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:05:14.276396 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.276373 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dc5f4dd74-mwpfj" Jul 24 15:05:14.278835 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.278810 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 24 15:05:14.278974 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.278836 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-v7hkw\"" Jul 24 15:05:14.279161 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.279142 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 24 15:05:14.285809 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.285604 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7dc5f4dd74-mwpfj"] Jul 24 15:05:14.336169 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.336129 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:05:14.428782 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.428746 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0bfb247-2f24-48e6-a076-9ae0ff67213e-webhook-cert\") pod \"maas-controller-7dc5f4dd74-mwpfj\" (UID: \"b0bfb247-2f24-48e6-a076-9ae0ff67213e\") " pod="opendatahub/maas-controller-7dc5f4dd74-mwpfj" Jul 24 15:05:14.428972 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.428815 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spgth\" (UniqueName: \"kubernetes.io/projected/b0bfb247-2f24-48e6-a076-9ae0ff67213e-kube-api-access-spgth\") pod \"maas-controller-7dc5f4dd74-mwpfj\" (UID: \"b0bfb247-2f24-48e6-a076-9ae0ff67213e\") " pod="opendatahub/maas-controller-7dc5f4dd74-mwpfj" Jul 24 15:05:14.436621 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.436591 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:05:14.457922 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.457892 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d8bd4bd76-46dr7"] Jul 24 15:05:14.460546 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.460524 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d8bd4bd76-46dr7" Jul 24 15:05:14.470999 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.470971 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d8bd4bd76-46dr7"] Jul 24 15:05:14.530697 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.530598 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0bfb247-2f24-48e6-a076-9ae0ff67213e-webhook-cert\") pod \"maas-controller-7dc5f4dd74-mwpfj\" (UID: \"b0bfb247-2f24-48e6-a076-9ae0ff67213e\") " pod="opendatahub/maas-controller-7dc5f4dd74-mwpfj" Jul 24 15:05:14.531047 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.531023 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77fp4\" (UniqueName: \"kubernetes.io/projected/88c38c4e-51c6-44fa-8f62-dce6cbfc8569-kube-api-access-77fp4\") pod \"maas-controller-7d8bd4bd76-46dr7\" (UID: \"88c38c4e-51c6-44fa-8f62-dce6cbfc8569\") " pod="opendatahub/maas-controller-7d8bd4bd76-46dr7" Jul 24 15:05:14.531204 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.531188 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/88c38c4e-51c6-44fa-8f62-dce6cbfc8569-webhook-cert\") pod \"maas-controller-7d8bd4bd76-46dr7\" (UID: \"88c38c4e-51c6-44fa-8f62-dce6cbfc8569\") " pod="opendatahub/maas-controller-7d8bd4bd76-46dr7" Jul 24 15:05:14.531341 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.531326 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spgth\" (UniqueName: \"kubernetes.io/projected/b0bfb247-2f24-48e6-a076-9ae0ff67213e-kube-api-access-spgth\") pod \"maas-controller-7dc5f4dd74-mwpfj\" (UID: \"b0bfb247-2f24-48e6-a076-9ae0ff67213e\") " pod="opendatahub/maas-controller-7dc5f4dd74-mwpfj" Jul 24 15:05:14.538837 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.538813 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0bfb247-2f24-48e6-a076-9ae0ff67213e-webhook-cert\") pod \"maas-controller-7dc5f4dd74-mwpfj\" (UID: \"b0bfb247-2f24-48e6-a076-9ae0ff67213e\") " pod="opendatahub/maas-controller-7dc5f4dd74-mwpfj" Jul 24 15:05:14.542430 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.542403 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spgth\" (UniqueName: \"kubernetes.io/projected/b0bfb247-2f24-48e6-a076-9ae0ff67213e-kube-api-access-spgth\") pod \"maas-controller-7dc5f4dd74-mwpfj\" (UID: \"b0bfb247-2f24-48e6-a076-9ae0ff67213e\") " pod="opendatahub/maas-controller-7dc5f4dd74-mwpfj" Jul 24 15:05:14.590141 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.590102 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dc5f4dd74-mwpfj" Jul 24 15:05:14.632959 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.632921 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77fp4\" (UniqueName: \"kubernetes.io/projected/88c38c4e-51c6-44fa-8f62-dce6cbfc8569-kube-api-access-77fp4\") pod \"maas-controller-7d8bd4bd76-46dr7\" (UID: \"88c38c4e-51c6-44fa-8f62-dce6cbfc8569\") " pod="opendatahub/maas-controller-7d8bd4bd76-46dr7" Jul 24 15:05:14.632959 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.632964 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/88c38c4e-51c6-44fa-8f62-dce6cbfc8569-webhook-cert\") pod \"maas-controller-7d8bd4bd76-46dr7\" (UID: \"88c38c4e-51c6-44fa-8f62-dce6cbfc8569\") " pod="opendatahub/maas-controller-7d8bd4bd76-46dr7" Jul 24 15:05:14.635503 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.635483 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/88c38c4e-51c6-44fa-8f62-dce6cbfc8569-webhook-cert\") pod \"maas-controller-7d8bd4bd76-46dr7\" (UID: \"88c38c4e-51c6-44fa-8f62-dce6cbfc8569\") " pod="opendatahub/maas-controller-7d8bd4bd76-46dr7" Jul 24 15:05:14.642742 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.642492 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77fp4\" (UniqueName: \"kubernetes.io/projected/88c38c4e-51c6-44fa-8f62-dce6cbfc8569-kube-api-access-77fp4\") pod \"maas-controller-7d8bd4bd76-46dr7\" (UID: \"88c38c4e-51c6-44fa-8f62-dce6cbfc8569\") " pod="opendatahub/maas-controller-7d8bd4bd76-46dr7" Jul 24 15:05:14.736268 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.736243 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7dc5f4dd74-mwpfj"] Jul 24 15:05:14.737902 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:05:14.737831 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0bfb247_2f24_48e6_a076_9ae0ff67213e.slice/crio-43a7cfc9484500fbd487cbe2fd29ab04c895807f8806c0d36868144b4d72a2dd WatchSource:0}: Error finding container 43a7cfc9484500fbd487cbe2fd29ab04c895807f8806c0d36868144b4d72a2dd: Status 404 returned error can't find the container with id 43a7cfc9484500fbd487cbe2fd29ab04c895807f8806c0d36868144b4d72a2dd Jul 24 15:05:14.772305 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.772280 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d8bd4bd76-46dr7" Jul 24 15:05:14.914771 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:14.914740 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d8bd4bd76-46dr7"] Jul 24 15:05:14.916349 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:05:14.916317 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88c38c4e_51c6_44fa_8f62_dce6cbfc8569.slice/crio-ff1dc87256e2956a975a61ecba74094118b77487c5d05e840d8422f8330b89b1 WatchSource:0}: Error finding container ff1dc87256e2956a975a61ecba74094118b77487c5d05e840d8422f8330b89b1: Status 404 returned error can't find the container with id ff1dc87256e2956a975a61ecba74094118b77487c5d05e840d8422f8330b89b1 Jul 24 15:05:15.258156 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:15.258121 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d8bd4bd76-46dr7" event={"ID":"88c38c4e-51c6-44fa-8f62-dce6cbfc8569","Type":"ContainerStarted","Data":"ff1dc87256e2956a975a61ecba74094118b77487c5d05e840d8422f8330b89b1"} Jul 24 15:05:15.259168 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:15.259147 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dc5f4dd74-mwpfj" event={"ID":"b0bfb247-2f24-48e6-a076-9ae0ff67213e","Type":"ContainerStarted","Data":"43a7cfc9484500fbd487cbe2fd29ab04c895807f8806c0d36868144b4d72a2dd"} Jul 24 15:05:17.573488 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:17.573455 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:05:17.655285 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:17.655196 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:05:18.284479 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:18.284445 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d8bd4bd76-46dr7" event={"ID":"88c38c4e-51c6-44fa-8f62-dce6cbfc8569","Type":"ContainerStarted","Data":"97111fe0a7e8c39c53ed38408be861aa20d439dbde1261fee203af02529409fe"} Jul 24 15:05:18.284763 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:18.284503 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d8bd4bd76-46dr7" Jul 24 15:05:18.285983 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:18.285955 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dc5f4dd74-mwpfj" event={"ID":"b0bfb247-2f24-48e6-a076-9ae0ff67213e","Type":"ContainerStarted","Data":"94c7d73c6da8eeda3b7929d6815d9122bd93193ad3594a5ed03572527e4f0bf1"} Jul 24 15:05:18.286094 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:18.286001 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7dc5f4dd74-mwpfj" Jul 24 15:05:18.304248 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:18.304203 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7d8bd4bd76-46dr7" podStartSLOduration=1.8355184759999998 podStartE2EDuration="4.304184799s" totalImagesPullingTime="2.468666323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:05:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:05:18.303106968 +0000 UTC m=+550.533583657" watchObservedRunningTime="2026-07-24 15:05:18.304184799 +0000 UTC m=+550.534661483" Jul 24 15:05:18.326884 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:18.326454 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7dc5f4dd74-mwpfj" podStartSLOduration=1.6895842829999999 podStartE2EDuration="4.326436876s" totalImagesPullingTime="2.636852593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:05:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:05:18.323486164 +0000 UTC m=+550.553962880" watchObservedRunningTime="2026-07-24 15:05:18.326436876 +0000 UTC m=+550.556913565" Jul 24 15:05:29.294542 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:29.294505 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d8bd4bd76-46dr7" Jul 24 15:05:29.295001 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:29.294623 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7dc5f4dd74-mwpfj" Jul 24 15:05:29.356883 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:29.356844 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7dc5f4dd74-mwpfj"] Jul 24 15:05:29.357095 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:29.357068 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7dc5f4dd74-mwpfj" podUID="b0bfb247-2f24-48e6-a076-9ae0ff67213e" containerName="manager" containerID="cri-o://94c7d73c6da8eeda3b7929d6815d9122bd93193ad3594a5ed03572527e4f0bf1" gracePeriod=30 Jul 24 15:05:29.610013 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:29.609985 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dc5f4dd74-mwpfj" Jul 24 15:05:29.769534 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:29.769495 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0bfb247-2f24-48e6-a076-9ae0ff67213e-webhook-cert\") pod \"b0bfb247-2f24-48e6-a076-9ae0ff67213e\" (UID: \"b0bfb247-2f24-48e6-a076-9ae0ff67213e\") " Jul 24 15:05:29.769698 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:29.769579 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spgth\" (UniqueName: \"kubernetes.io/projected/b0bfb247-2f24-48e6-a076-9ae0ff67213e-kube-api-access-spgth\") pod \"b0bfb247-2f24-48e6-a076-9ae0ff67213e\" (UID: \"b0bfb247-2f24-48e6-a076-9ae0ff67213e\") " Jul 24 15:05:29.771899 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:29.771848 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0bfb247-2f24-48e6-a076-9ae0ff67213e-kube-api-access-spgth" (OuterVolumeSpecName: "kube-api-access-spgth") pod "b0bfb247-2f24-48e6-a076-9ae0ff67213e" (UID: "b0bfb247-2f24-48e6-a076-9ae0ff67213e"). InnerVolumeSpecName "kube-api-access-spgth". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:05:29.771899 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:29.771888 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0bfb247-2f24-48e6-a076-9ae0ff67213e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b0bfb247-2f24-48e6-a076-9ae0ff67213e" (UID: "b0bfb247-2f24-48e6-a076-9ae0ff67213e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:05:29.870320 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:29.870249 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spgth\" (UniqueName: \"kubernetes.io/projected/b0bfb247-2f24-48e6-a076-9ae0ff67213e-kube-api-access-spgth\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:05:29.870320 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:29.870271 2531 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0bfb247-2f24-48e6-a076-9ae0ff67213e-webhook-cert\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:05:30.355299 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:30.355265 2531 generic.go:356] "Generic (PLEG): container finished" podID="b0bfb247-2f24-48e6-a076-9ae0ff67213e" containerID="94c7d73c6da8eeda3b7929d6815d9122bd93193ad3594a5ed03572527e4f0bf1" exitCode=0 Jul 24 15:05:30.355657 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:30.355320 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dc5f4dd74-mwpfj" Jul 24 15:05:30.355657 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:30.355359 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dc5f4dd74-mwpfj" event={"ID":"b0bfb247-2f24-48e6-a076-9ae0ff67213e","Type":"ContainerDied","Data":"94c7d73c6da8eeda3b7929d6815d9122bd93193ad3594a5ed03572527e4f0bf1"} Jul 24 15:05:30.355657 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:30.355419 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dc5f4dd74-mwpfj" event={"ID":"b0bfb247-2f24-48e6-a076-9ae0ff67213e","Type":"ContainerDied","Data":"43a7cfc9484500fbd487cbe2fd29ab04c895807f8806c0d36868144b4d72a2dd"} Jul 24 15:05:30.355657 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:30.355443 2531 scope.go:117] "RemoveContainer" containerID="94c7d73c6da8eeda3b7929d6815d9122bd93193ad3594a5ed03572527e4f0bf1" Jul 24 15:05:30.367513 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:30.367319 2531 scope.go:117] "RemoveContainer" containerID="94c7d73c6da8eeda3b7929d6815d9122bd93193ad3594a5ed03572527e4f0bf1" Jul 24 15:05:30.367568 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:05:30.367546 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94c7d73c6da8eeda3b7929d6815d9122bd93193ad3594a5ed03572527e4f0bf1\": container with ID starting with 94c7d73c6da8eeda3b7929d6815d9122bd93193ad3594a5ed03572527e4f0bf1 not found: ID does not exist" containerID="94c7d73c6da8eeda3b7929d6815d9122bd93193ad3594a5ed03572527e4f0bf1" Jul 24 15:05:30.367606 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:30.367573 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94c7d73c6da8eeda3b7929d6815d9122bd93193ad3594a5ed03572527e4f0bf1"} err="failed to get container status \"94c7d73c6da8eeda3b7929d6815d9122bd93193ad3594a5ed03572527e4f0bf1\": rpc error: code = NotFound desc = could not find container \"94c7d73c6da8eeda3b7929d6815d9122bd93193ad3594a5ed03572527e4f0bf1\": container with ID starting with 94c7d73c6da8eeda3b7929d6815d9122bd93193ad3594a5ed03572527e4f0bf1 not found: ID does not exist" Jul 24 15:05:30.374419 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:30.374396 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7dc5f4dd74-mwpfj"] Jul 24 15:05:30.378117 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:30.378096 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7dc5f4dd74-mwpfj"] Jul 24 15:05:32.277882 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:32.277834 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0bfb247-2f24-48e6-a076-9ae0ff67213e" path="/var/lib/kubelet/pods/b0bfb247-2f24-48e6-a076-9ae0ff67213e/volumes" Jul 24 15:05:49.300339 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:49.300304 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:05:49.402091 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:49.402055 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:05:50.019478 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.019442 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:05:50.077107 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.077070 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-64775d5d55-9xvvz"] Jul 24 15:05:50.077491 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.077479 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0bfb247-2f24-48e6-a076-9ae0ff67213e" containerName="manager" Jul 24 15:05:50.077535 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.077493 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0bfb247-2f24-48e6-a076-9ae0ff67213e" containerName="manager" Jul 24 15:05:50.077570 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.077559 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0bfb247-2f24-48e6-a076-9ae0ff67213e" containerName="manager" Jul 24 15:05:50.082474 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.082456 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-64775d5d55-9xvvz" Jul 24 15:05:50.085072 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.085051 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 15:05:50.085180 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.085152 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 24 15:05:50.086012 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.085998 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 24 15:05:50.094694 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.094671 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-64775d5d55-9xvvz"] Jul 24 15:05:50.125573 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.125548 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:05:50.134709 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.134682 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjb2n\" (UniqueName: \"kubernetes.io/projected/e4f641a2-ee07-4901-80cd-ab54a50faebf-kube-api-access-vjb2n\") pod \"maas-api-64775d5d55-9xvvz\" (UID: \"e4f641a2-ee07-4901-80cd-ab54a50faebf\") " pod="odh-ai-gateway-infra/maas-api-64775d5d55-9xvvz" Jul 24 15:05:50.134826 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.134742 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4f641a2-ee07-4901-80cd-ab54a50faebf-maas-api-tls\") pod \"maas-api-64775d5d55-9xvvz\" (UID: \"e4f641a2-ee07-4901-80cd-ab54a50faebf\") " pod="odh-ai-gateway-infra/maas-api-64775d5d55-9xvvz" Jul 24 15:05:50.134941 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.134920 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f641a2-ee07-4901-80cd-ab54a50faebf-openshift-service-ca\") pod \"maas-api-64775d5d55-9xvvz\" (UID: \"e4f641a2-ee07-4901-80cd-ab54a50faebf\") " pod="odh-ai-gateway-infra/maas-api-64775d5d55-9xvvz" Jul 24 15:05:50.235846 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.235813 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjb2n\" (UniqueName: \"kubernetes.io/projected/e4f641a2-ee07-4901-80cd-ab54a50faebf-kube-api-access-vjb2n\") pod \"maas-api-64775d5d55-9xvvz\" (UID: \"e4f641a2-ee07-4901-80cd-ab54a50faebf\") " pod="odh-ai-gateway-infra/maas-api-64775d5d55-9xvvz" Jul 24 15:05:50.236060 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.235854 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4f641a2-ee07-4901-80cd-ab54a50faebf-maas-api-tls\") pod \"maas-api-64775d5d55-9xvvz\" (UID: \"e4f641a2-ee07-4901-80cd-ab54a50faebf\") " pod="odh-ai-gateway-infra/maas-api-64775d5d55-9xvvz" Jul 24 15:05:50.236060 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.235961 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f641a2-ee07-4901-80cd-ab54a50faebf-openshift-service-ca\") pod \"maas-api-64775d5d55-9xvvz\" (UID: \"e4f641a2-ee07-4901-80cd-ab54a50faebf\") " pod="odh-ai-gateway-infra/maas-api-64775d5d55-9xvvz" Jul 24 15:05:50.236668 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.236649 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f641a2-ee07-4901-80cd-ab54a50faebf-openshift-service-ca\") pod \"maas-api-64775d5d55-9xvvz\" (UID: \"e4f641a2-ee07-4901-80cd-ab54a50faebf\") " pod="odh-ai-gateway-infra/maas-api-64775d5d55-9xvvz" Jul 24 15:05:50.238439 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.238410 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4f641a2-ee07-4901-80cd-ab54a50faebf-maas-api-tls\") pod \"maas-api-64775d5d55-9xvvz\" (UID: \"e4f641a2-ee07-4901-80cd-ab54a50faebf\") " pod="odh-ai-gateway-infra/maas-api-64775d5d55-9xvvz" Jul 24 15:05:50.245398 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.245372 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjb2n\" (UniqueName: \"kubernetes.io/projected/e4f641a2-ee07-4901-80cd-ab54a50faebf-kube-api-access-vjb2n\") pod \"maas-api-64775d5d55-9xvvz\" (UID: \"e4f641a2-ee07-4901-80cd-ab54a50faebf\") " pod="odh-ai-gateway-infra/maas-api-64775d5d55-9xvvz" Jul 24 15:05:50.393908 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.393800 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-64775d5d55-9xvvz" Jul 24 15:05:50.542328 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.542302 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-64775d5d55-9xvvz"] Jul 24 15:05:50.544068 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:05:50.544037 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4f641a2_ee07_4901_80cd_ab54a50faebf.slice/crio-f54d674c813920cc39ed176ac34abf8ef31b26051af01e3687d797007c4449a4 WatchSource:0}: Error finding container f54d674c813920cc39ed176ac34abf8ef31b26051af01e3687d797007c4449a4: Status 404 returned error can't find the container with id f54d674c813920cc39ed176ac34abf8ef31b26051af01e3687d797007c4449a4 Jul 24 15:05:50.561590 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.561557 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:05:50.665765 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.665666 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:05:50.692494 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.692465 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:05:50.719881 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.719839 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-hfcxx"] Jul 24 15:05:50.723243 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.723219 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-hfcxx" Jul 24 15:05:50.725752 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.725728 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 24 15:05:50.725919 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.725825 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-fv8dh\"" Jul 24 15:05:50.735256 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.735235 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-hfcxx"] Jul 24 15:05:50.842230 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.842192 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zsnl\" (UniqueName: \"kubernetes.io/projected/58138eae-76c2-489f-abdb-f858b5ef0108-kube-api-access-8zsnl\") pod \"payload-processing-5cc6cd444b-hfcxx\" (UID: \"58138eae-76c2-489f-abdb-f858b5ef0108\") " pod="openshift-ingress/payload-processing-5cc6cd444b-hfcxx" Jul 24 15:05:50.842421 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.842242 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58138eae-76c2-489f-abdb-f858b5ef0108-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-hfcxx\" (UID: \"58138eae-76c2-489f-abdb-f858b5ef0108\") " pod="openshift-ingress/payload-processing-5cc6cd444b-hfcxx" Jul 24 15:05:50.943255 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.943227 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zsnl\" (UniqueName: \"kubernetes.io/projected/58138eae-76c2-489f-abdb-f858b5ef0108-kube-api-access-8zsnl\") pod \"payload-processing-5cc6cd444b-hfcxx\" (UID: \"58138eae-76c2-489f-abdb-f858b5ef0108\") " pod="openshift-ingress/payload-processing-5cc6cd444b-hfcxx" Jul 24 15:05:50.943455 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.943267 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58138eae-76c2-489f-abdb-f858b5ef0108-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-hfcxx\" (UID: \"58138eae-76c2-489f-abdb-f858b5ef0108\") " pod="openshift-ingress/payload-processing-5cc6cd444b-hfcxx" Jul 24 15:05:50.944054 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.944035 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58138eae-76c2-489f-abdb-f858b5ef0108-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-hfcxx\" (UID: \"58138eae-76c2-489f-abdb-f858b5ef0108\") " pod="openshift-ingress/payload-processing-5cc6cd444b-hfcxx" Jul 24 15:05:50.952226 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:50.952201 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zsnl\" (UniqueName: \"kubernetes.io/projected/58138eae-76c2-489f-abdb-f858b5ef0108-kube-api-access-8zsnl\") pod \"payload-processing-5cc6cd444b-hfcxx\" (UID: \"58138eae-76c2-489f-abdb-f858b5ef0108\") " pod="openshift-ingress/payload-processing-5cc6cd444b-hfcxx" Jul 24 15:05:51.034926 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:51.034885 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-hfcxx" Jul 24 15:05:51.183350 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:51.183321 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-hfcxx"] Jul 24 15:05:51.185560 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:05:51.185529 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58138eae_76c2_489f_abdb_f858b5ef0108.slice/crio-ea96c23f1a312c45f0c4106756b71fdd4a9e8d61d352bd919590dc5f637bf4df WatchSource:0}: Error finding container ea96c23f1a312c45f0c4106756b71fdd4a9e8d61d352bd919590dc5f637bf4df: Status 404 returned error can't find the container with id ea96c23f1a312c45f0c4106756b71fdd4a9e8d61d352bd919590dc5f637bf4df Jul 24 15:05:51.474534 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:51.474497 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-hfcxx" event={"ID":"58138eae-76c2-489f-abdb-f858b5ef0108","Type":"ContainerStarted","Data":"ea96c23f1a312c45f0c4106756b71fdd4a9e8d61d352bd919590dc5f637bf4df"} Jul 24 15:05:51.475593 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:51.475570 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-64775d5d55-9xvvz" event={"ID":"e4f641a2-ee07-4901-80cd-ab54a50faebf","Type":"ContainerStarted","Data":"f54d674c813920cc39ed176ac34abf8ef31b26051af01e3687d797007c4449a4"} Jul 24 15:05:53.522223 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:53.522129 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-hfcxx" event={"ID":"58138eae-76c2-489f-abdb-f858b5ef0108","Type":"ContainerStarted","Data":"d7a8b6f0badf1e29c74950d07069af01998dfd303f6b9e0f6d1d5218fbb54b92"} Jul 24 15:05:53.522705 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:53.522330 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-hfcxx" Jul 24 15:05:53.524015 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:53.523987 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-64775d5d55-9xvvz" event={"ID":"e4f641a2-ee07-4901-80cd-ab54a50faebf","Type":"ContainerStarted","Data":"5d1534dc51309586ab040e4f80de00198664fa7003c4f6e068c8d66e39880db3"} Jul 24 15:05:53.524150 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:53.524115 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-64775d5d55-9xvvz" Jul 24 15:05:53.537133 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:53.537080 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-hfcxx" podStartSLOduration=1.480065823 podStartE2EDuration="3.537063891s" totalImagesPullingTime="2.056998068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:05:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:05:53.536588172 +0000 UTC m=+585.767064874" watchObservedRunningTime="2026-07-24 15:05:53.537063891 +0000 UTC m=+585.767540577" Jul 24 15:05:53.553269 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:53.553224 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-64775d5d55-9xvvz" podStartSLOduration=1.439348161 podStartE2EDuration="3.553211435s" totalImagesPullingTime="2.113863274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:05:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:05:53.552826975 +0000 UTC m=+585.783303661" watchObservedRunningTime="2026-07-24 15:05:53.553211435 +0000 UTC m=+585.783688120" Jul 24 15:05:58.557655 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:58.557617 2531 generic.go:356] "Generic (PLEG): container finished" podID="0e873231-3e07-4d1d-9d0d-bf7cb190d9c3" containerID="4c6465b05df96203d5bb98a94fe3e202761c0bf73d11bbbb20aec426b8073eb1" exitCode=0 Jul 24 15:05:58.558106 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:58.557689 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" event={"ID":"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3","Type":"ContainerDied","Data":"4c6465b05df96203d5bb98a94fe3e202761c0bf73d11bbbb20aec426b8073eb1"} Jul 24 15:05:58.558106 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:58.558081 2531 scope.go:117] "RemoveContainer" containerID="4c6465b05df96203d5bb98a94fe3e202761c0bf73d11bbbb20aec426b8073eb1" Jul 24 15:05:58.559140 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:58.559116 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 24 15:05:58.559209 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:58.559178 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 24 15:05:58.559248 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:58.559209 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 24 15:05:59.537099 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:59.537068 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-64775d5d55-9xvvz" Jul 24 15:05:59.567244 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:59.567203 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" event={"ID":"0e873231-3e07-4d1d-9d0d-bf7cb190d9c3","Type":"ContainerStarted","Data":"3e1e4b31771ce73edc2787f601758ff4233da86d63447d29069112fb7bc9ee33"} Jul 24 15:05:59.839357 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:59.839248 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:05:59.840486 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:59.840457 2531 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-gzdst container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 24 15:05:59.840626 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:05:59.840509 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" podUID="0e873231-3e07-4d1d-9d0d-bf7cb190d9c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 24 15:06:00.593693 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:00.593652 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:06:00.694836 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:00.694797 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:06:00.838816 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:00.838782 2531 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-gzdst container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 24 15:06:00.839001 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:00.838842 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" podUID="0e873231-3e07-4d1d-9d0d-bf7cb190d9c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 24 15:06:00.986436 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:00.986388 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:06:01.839205 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:01.839166 2531 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-gzdst container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 24 15:06:01.839701 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:01.839225 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" podUID="0e873231-3e07-4d1d-9d0d-bf7cb190d9c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 24 15:06:02.838684 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:02.838647 2531 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-gzdst container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 24 15:06:02.838851 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:02.838701 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" podUID="0e873231-3e07-4d1d-9d0d-bf7cb190d9c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 24 15:06:03.843150 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:03.843118 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:06:03.843897 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:03.843880 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-gzdst" Jul 24 15:06:04.533620 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:04.533581 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-hfcxx" Jul 24 15:06:05.652800 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.652768 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q"] Jul 24 15:06:05.655762 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.655743 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:05.659183 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.659163 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mx8dd\"" Jul 24 15:06:05.659297 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.659163 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 24 15:06:05.659472 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.659450 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 24 15:06:05.659543 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.659524 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 24 15:06:05.670809 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.670788 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q"] Jul 24 15:06:05.699924 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.699890 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:06:05.792720 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.792690 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/407c4619-b7cb-43d2-8ab9-d15fead0de5e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q\" (UID: \"407c4619-b7cb-43d2-8ab9-d15fead0de5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:05.792903 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.792727 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/407c4619-b7cb-43d2-8ab9-d15fead0de5e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q\" (UID: \"407c4619-b7cb-43d2-8ab9-d15fead0de5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:05.792903 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.792757 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/407c4619-b7cb-43d2-8ab9-d15fead0de5e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q\" (UID: \"407c4619-b7cb-43d2-8ab9-d15fead0de5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:05.793018 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.792902 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnplt\" (UniqueName: \"kubernetes.io/projected/407c4619-b7cb-43d2-8ab9-d15fead0de5e-kube-api-access-mnplt\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q\" (UID: \"407c4619-b7cb-43d2-8ab9-d15fead0de5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:05.793018 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.792932 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/407c4619-b7cb-43d2-8ab9-d15fead0de5e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q\" (UID: \"407c4619-b7cb-43d2-8ab9-d15fead0de5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:05.793092 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.793065 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/407c4619-b7cb-43d2-8ab9-d15fead0de5e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q\" (UID: \"407c4619-b7cb-43d2-8ab9-d15fead0de5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:05.894221 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.894194 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/407c4619-b7cb-43d2-8ab9-d15fead0de5e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q\" (UID: \"407c4619-b7cb-43d2-8ab9-d15fead0de5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:05.894393 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.894259 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/407c4619-b7cb-43d2-8ab9-d15fead0de5e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q\" (UID: \"407c4619-b7cb-43d2-8ab9-d15fead0de5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:05.894393 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.894311 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/407c4619-b7cb-43d2-8ab9-d15fead0de5e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q\" (UID: \"407c4619-b7cb-43d2-8ab9-d15fead0de5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:05.894393 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.894328 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/407c4619-b7cb-43d2-8ab9-d15fead0de5e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q\" (UID: \"407c4619-b7cb-43d2-8ab9-d15fead0de5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:05.894393 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.894349 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/407c4619-b7cb-43d2-8ab9-d15fead0de5e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q\" (UID: \"407c4619-b7cb-43d2-8ab9-d15fead0de5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:05.894596 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.894431 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnplt\" (UniqueName: \"kubernetes.io/projected/407c4619-b7cb-43d2-8ab9-d15fead0de5e-kube-api-access-mnplt\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q\" (UID: \"407c4619-b7cb-43d2-8ab9-d15fead0de5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:05.894657 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.894593 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/407c4619-b7cb-43d2-8ab9-d15fead0de5e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q\" (UID: \"407c4619-b7cb-43d2-8ab9-d15fead0de5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:05.894712 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.894663 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/407c4619-b7cb-43d2-8ab9-d15fead0de5e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q\" (UID: \"407c4619-b7cb-43d2-8ab9-d15fead0de5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:05.894712 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.894701 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/407c4619-b7cb-43d2-8ab9-d15fead0de5e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q\" (UID: \"407c4619-b7cb-43d2-8ab9-d15fead0de5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:05.896802 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.896782 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/407c4619-b7cb-43d2-8ab9-d15fead0de5e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q\" (UID: \"407c4619-b7cb-43d2-8ab9-d15fead0de5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:05.897199 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.897178 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/407c4619-b7cb-43d2-8ab9-d15fead0de5e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q\" (UID: \"407c4619-b7cb-43d2-8ab9-d15fead0de5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:05.905537 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.905469 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnplt\" (UniqueName: \"kubernetes.io/projected/407c4619-b7cb-43d2-8ab9-d15fead0de5e-kube-api-access-mnplt\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q\" (UID: \"407c4619-b7cb-43d2-8ab9-d15fead0de5e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:05.967307 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:05.967285 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:06.135169 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:06.135142 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q"] Jul 24 15:06:06.137175 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:06:06.137150 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod407c4619_b7cb_43d2_8ab9_d15fead0de5e.slice/crio-87ae5c30974f99d0dd2060c0a8759ef315ce09f20259349f68323daf19b32e46 WatchSource:0}: Error finding container 87ae5c30974f99d0dd2060c0a8759ef315ce09f20259349f68323daf19b32e46: Status 404 returned error can't find the container with id 87ae5c30974f99d0dd2060c0a8759ef315ce09f20259349f68323daf19b32e46 Jul 24 15:06:06.176097 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:06.176066 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:06:06.616820 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:06.616728 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" event={"ID":"407c4619-b7cb-43d2-8ab9-d15fead0de5e","Type":"ContainerStarted","Data":"87ae5c30974f99d0dd2060c0a8759ef315ce09f20259349f68323daf19b32e46"} Jul 24 15:06:07.624692 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:07.624607 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" event={"ID":"407c4619-b7cb-43d2-8ab9-d15fead0de5e","Type":"ContainerStarted","Data":"72bca582ceb34c66c40e2c03a559d2c62a1b93cf71a202c272791a80b9172188"} Jul 24 15:06:07.647016 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:07.646965 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" podStartSLOduration=1.587414428 podStartE2EDuration="2.646951124s" totalImagesPullingTime="1.059536696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:06:07.64468724 +0000 UTC m=+599.875163944" watchObservedRunningTime="2026-07-24 15:06:07.646951124 +0000 UTC m=+599.877427808" Jul 24 15:06:08.200210 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:08.200178 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:06:08.201150 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:08.201130 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:06:14.549702 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:14.549653 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:06:15.354573 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:15.354535 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:06:15.968392 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:15.968354 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:15.968826 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:15.968430 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:15.981072 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:15.981049 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:16.686760 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:16.686727 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q" Jul 24 15:06:24.189418 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:24.189380 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:06:25.669794 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:25.669756 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:06:27.026767 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.026732 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs"] Jul 24 15:06:27.030302 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.030282 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:27.032915 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.032889 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 24 15:06:27.040659 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.040634 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs"] Jul 24 15:06:27.162125 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.162090 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:06:27.186663 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.186628 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed14cd07-72ec-4afa-a66b-11292da5ddb0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs\" (UID: \"ed14cd07-72ec-4afa-a66b-11292da5ddb0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:27.186802 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.186753 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed14cd07-72ec-4afa-a66b-11292da5ddb0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs\" (UID: \"ed14cd07-72ec-4afa-a66b-11292da5ddb0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:27.186802 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.186787 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed14cd07-72ec-4afa-a66b-11292da5ddb0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs\" (UID: \"ed14cd07-72ec-4afa-a66b-11292da5ddb0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:27.186912 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.186815 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed14cd07-72ec-4afa-a66b-11292da5ddb0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs\" (UID: \"ed14cd07-72ec-4afa-a66b-11292da5ddb0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:27.186952 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.186922 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt9zl\" (UniqueName: \"kubernetes.io/projected/ed14cd07-72ec-4afa-a66b-11292da5ddb0-kube-api-access-pt9zl\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs\" (UID: \"ed14cd07-72ec-4afa-a66b-11292da5ddb0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:27.186988 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.186968 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed14cd07-72ec-4afa-a66b-11292da5ddb0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs\" (UID: \"ed14cd07-72ec-4afa-a66b-11292da5ddb0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:27.287599 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.287522 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed14cd07-72ec-4afa-a66b-11292da5ddb0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs\" (UID: \"ed14cd07-72ec-4afa-a66b-11292da5ddb0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:27.287599 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.287575 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed14cd07-72ec-4afa-a66b-11292da5ddb0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs\" (UID: \"ed14cd07-72ec-4afa-a66b-11292da5ddb0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:27.287798 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.287613 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed14cd07-72ec-4afa-a66b-11292da5ddb0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs\" (UID: \"ed14cd07-72ec-4afa-a66b-11292da5ddb0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:27.287798 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.287663 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt9zl\" (UniqueName: \"kubernetes.io/projected/ed14cd07-72ec-4afa-a66b-11292da5ddb0-kube-api-access-pt9zl\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs\" (UID: \"ed14cd07-72ec-4afa-a66b-11292da5ddb0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:27.287798 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.287713 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed14cd07-72ec-4afa-a66b-11292da5ddb0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs\" (UID: \"ed14cd07-72ec-4afa-a66b-11292da5ddb0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:27.287798 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.287740 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed14cd07-72ec-4afa-a66b-11292da5ddb0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs\" (UID: \"ed14cd07-72ec-4afa-a66b-11292da5ddb0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:27.288121 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.288091 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed14cd07-72ec-4afa-a66b-11292da5ddb0-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs\" (UID: \"ed14cd07-72ec-4afa-a66b-11292da5ddb0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:27.288198 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.288130 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed14cd07-72ec-4afa-a66b-11292da5ddb0-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs\" (UID: \"ed14cd07-72ec-4afa-a66b-11292da5ddb0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:27.288198 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.288179 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed14cd07-72ec-4afa-a66b-11292da5ddb0-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs\" (UID: \"ed14cd07-72ec-4afa-a66b-11292da5ddb0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:27.290548 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.290525 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed14cd07-72ec-4afa-a66b-11292da5ddb0-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs\" (UID: \"ed14cd07-72ec-4afa-a66b-11292da5ddb0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:27.290797 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.290781 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed14cd07-72ec-4afa-a66b-11292da5ddb0-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs\" (UID: \"ed14cd07-72ec-4afa-a66b-11292da5ddb0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:27.296462 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.296444 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt9zl\" (UniqueName: \"kubernetes.io/projected/ed14cd07-72ec-4afa-a66b-11292da5ddb0-kube-api-access-pt9zl\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs\" (UID: \"ed14cd07-72ec-4afa-a66b-11292da5ddb0\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:27.340599 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.340570 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:27.487660 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.487636 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs"] Jul 24 15:06:27.490445 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:06:27.490421 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded14cd07_72ec_4afa_a66b_11292da5ddb0.slice/crio-b7e7b900310a420ffb9e1cae333a97a90456c6b35f0af48a92a89c851b19c769 WatchSource:0}: Error finding container b7e7b900310a420ffb9e1cae333a97a90456c6b35f0af48a92a89c851b19c769: Status 404 returned error can't find the container with id b7e7b900310a420ffb9e1cae333a97a90456c6b35f0af48a92a89c851b19c769 Jul 24 15:06:27.492144 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.492127 2531 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 15:06:27.740928 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:27.740895 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" event={"ID":"ed14cd07-72ec-4afa-a66b-11292da5ddb0","Type":"ContainerStarted","Data":"b7e7b900310a420ffb9e1cae333a97a90456c6b35f0af48a92a89c851b19c769"} Jul 24 15:06:28.748391 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:28.748346 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" event={"ID":"ed14cd07-72ec-4afa-a66b-11292da5ddb0","Type":"ContainerStarted","Data":"378c57302a09cec71bff9beb5da13fc335733ef6e0f00572b420bd4f6356435a"} Jul 24 15:06:28.766491 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:28.766440 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" podStartSLOduration=1.557309752 podStartE2EDuration="1.766425158s" totalImagesPullingTime="209.115406ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:06:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:06:28.765807351 +0000 UTC m=+620.996284035" watchObservedRunningTime="2026-07-24 15:06:28.766425158 +0000 UTC m=+620.996901842" Jul 24 15:06:31.552450 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:31.552411 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:06:37.341116 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:37.341081 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:37.341511 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:37.341158 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:37.353311 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:37.353286 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:37.813628 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:37.813601 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs" Jul 24 15:06:39.576409 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:39.576370 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:06:42.359267 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:42.359225 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:06:56.996625 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:56.996583 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:06:57.068381 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:57.068349 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:06:59.978226 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:06:59.978192 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:07:02.574684 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:02.574650 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:07:14.796045 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:14.796006 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:07:23.881306 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:23.881267 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:07:23.945303 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:23.945269 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:07:28.459438 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:28.459399 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:07:57.490960 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:57.490923 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6954746ddf-mgsft"] Jul 24 15:07:57.493857 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:57.493836 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6954746ddf-mgsft" Jul 24 15:07:57.497033 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:57.497012 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 24 15:07:57.497144 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:57.497040 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hdm24\"" Jul 24 15:07:57.502143 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:57.502118 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6954746ddf-mgsft"] Jul 24 15:07:57.627387 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:57.627359 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvmch\" (UniqueName: \"kubernetes.io/projected/de5f106e-828b-4754-8440-61fa371ddc71-kube-api-access-fvmch\") pod \"authorino-6954746ddf-mgsft\" (UID: \"de5f106e-828b-4754-8440-61fa371ddc71\") " pod="kuadrant-system/authorino-6954746ddf-mgsft" Jul 24 15:07:57.627558 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:57.627402 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/de5f106e-828b-4754-8440-61fa371ddc71-tls-cert\") pod \"authorino-6954746ddf-mgsft\" (UID: \"de5f106e-828b-4754-8440-61fa371ddc71\") " pod="kuadrant-system/authorino-6954746ddf-mgsft" Jul 24 15:07:57.728636 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:57.728603 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvmch\" (UniqueName: \"kubernetes.io/projected/de5f106e-828b-4754-8440-61fa371ddc71-kube-api-access-fvmch\") pod \"authorino-6954746ddf-mgsft\" (UID: \"de5f106e-828b-4754-8440-61fa371ddc71\") " pod="kuadrant-system/authorino-6954746ddf-mgsft" Jul 24 15:07:57.728819 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:57.728649 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/de5f106e-828b-4754-8440-61fa371ddc71-tls-cert\") pod \"authorino-6954746ddf-mgsft\" (UID: \"de5f106e-828b-4754-8440-61fa371ddc71\") " pod="kuadrant-system/authorino-6954746ddf-mgsft" Jul 24 15:07:57.731185 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:57.731166 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/de5f106e-828b-4754-8440-61fa371ddc71-tls-cert\") pod \"authorino-6954746ddf-mgsft\" (UID: \"de5f106e-828b-4754-8440-61fa371ddc71\") " pod="kuadrant-system/authorino-6954746ddf-mgsft" Jul 24 15:07:57.736623 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:57.736605 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvmch\" (UniqueName: \"kubernetes.io/projected/de5f106e-828b-4754-8440-61fa371ddc71-kube-api-access-fvmch\") pod \"authorino-6954746ddf-mgsft\" (UID: \"de5f106e-828b-4754-8440-61fa371ddc71\") " pod="kuadrant-system/authorino-6954746ddf-mgsft" Jul 24 15:07:57.804815 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:57.804754 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6954746ddf-mgsft" Jul 24 15:07:57.954275 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:57.954246 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:07:57.963304 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:57.963278 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6954746ddf-mgsft"] Jul 24 15:07:57.965541 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:07:57.965511 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde5f106e_828b_4754_8440_61fa371ddc71.slice/crio-4dc42c933f1e2287a4e595f59a8c5629e2303aaa73a7d6443276791d231618c1 WatchSource:0}: Error finding container 4dc42c933f1e2287a4e595f59a8c5629e2303aaa73a7d6443276791d231618c1: Status 404 returned error can't find the container with id 4dc42c933f1e2287a4e595f59a8c5629e2303aaa73a7d6443276791d231618c1 Jul 24 15:07:58.104921 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:58.104801 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:07:58.287085 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:58.287055 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6954746ddf-mgsft" event={"ID":"de5f106e-828b-4754-8440-61fa371ddc71","Type":"ContainerStarted","Data":"4dc42c933f1e2287a4e595f59a8c5629e2303aaa73a7d6443276791d231618c1"} Jul 24 15:07:59.295222 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:59.295137 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6954746ddf-mgsft" event={"ID":"de5f106e-828b-4754-8440-61fa371ddc71","Type":"ContainerStarted","Data":"4d187443e86bafa8c82d876ac19f8d1f1e16f67d73a12a0d7d98d63cbfd0889b"} Jul 24 15:07:59.296673 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:59.296647 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:07:59.320219 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:59.320171 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6954746ddf-mgsft" podStartSLOduration=1.814846018 podStartE2EDuration="2.320154541s" totalImagesPullingTime="505.308523ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:07:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:07:59.318618408 +0000 UTC m=+711.549095091" watchObservedRunningTime="2026-07-24 15:07:59.320154541 +0000 UTC m=+711.550631242" Jul 24 15:07:59.404390 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:07:59.404355 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:08:28.881633 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:28.881594 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:08:38.472257 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:38.472220 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:08:46.161335 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:46.161304 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d8bd4bd76-46dr7"] Jul 24 15:08:46.161741 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:46.161497 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7d8bd4bd76-46dr7" podUID="88c38c4e-51c6-44fa-8f62-dce6cbfc8569" containerName="manager" containerID="cri-o://97111fe0a7e8c39c53ed38408be861aa20d439dbde1261fee203af02529409fe" gracePeriod=30 Jul 24 15:08:46.414002 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:46.413935 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d8bd4bd76-46dr7" Jul 24 15:08:46.492029 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:46.491996 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/88c38c4e-51c6-44fa-8f62-dce6cbfc8569-webhook-cert\") pod \"88c38c4e-51c6-44fa-8f62-dce6cbfc8569\" (UID: \"88c38c4e-51c6-44fa-8f62-dce6cbfc8569\") " Jul 24 15:08:46.492213 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:46.492044 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77fp4\" (UniqueName: \"kubernetes.io/projected/88c38c4e-51c6-44fa-8f62-dce6cbfc8569-kube-api-access-77fp4\") pod \"88c38c4e-51c6-44fa-8f62-dce6cbfc8569\" (UID: \"88c38c4e-51c6-44fa-8f62-dce6cbfc8569\") " Jul 24 15:08:46.494129 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:46.494099 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88c38c4e-51c6-44fa-8f62-dce6cbfc8569-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "88c38c4e-51c6-44fa-8f62-dce6cbfc8569" (UID: "88c38c4e-51c6-44fa-8f62-dce6cbfc8569"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:08:46.494247 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:46.494222 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88c38c4e-51c6-44fa-8f62-dce6cbfc8569-kube-api-access-77fp4" (OuterVolumeSpecName: "kube-api-access-77fp4") pod "88c38c4e-51c6-44fa-8f62-dce6cbfc8569" (UID: "88c38c4e-51c6-44fa-8f62-dce6cbfc8569"). InnerVolumeSpecName "kube-api-access-77fp4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:08:46.579702 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:46.579664 2531 generic.go:356] "Generic (PLEG): container finished" podID="88c38c4e-51c6-44fa-8f62-dce6cbfc8569" containerID="97111fe0a7e8c39c53ed38408be861aa20d439dbde1261fee203af02529409fe" exitCode=0 Jul 24 15:08:46.579903 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:46.579713 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d8bd4bd76-46dr7" Jul 24 15:08:46.579903 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:46.579759 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d8bd4bd76-46dr7" event={"ID":"88c38c4e-51c6-44fa-8f62-dce6cbfc8569","Type":"ContainerDied","Data":"97111fe0a7e8c39c53ed38408be861aa20d439dbde1261fee203af02529409fe"} Jul 24 15:08:46.579903 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:46.579820 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d8bd4bd76-46dr7" event={"ID":"88c38c4e-51c6-44fa-8f62-dce6cbfc8569","Type":"ContainerDied","Data":"ff1dc87256e2956a975a61ecba74094118b77487c5d05e840d8422f8330b89b1"} Jul 24 15:08:46.579903 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:46.579849 2531 scope.go:117] "RemoveContainer" containerID="97111fe0a7e8c39c53ed38408be861aa20d439dbde1261fee203af02529409fe" Jul 24 15:08:46.589991 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:46.589973 2531 scope.go:117] "RemoveContainer" containerID="97111fe0a7e8c39c53ed38408be861aa20d439dbde1261fee203af02529409fe" Jul 24 15:08:46.590219 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:08:46.590204 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97111fe0a7e8c39c53ed38408be861aa20d439dbde1261fee203af02529409fe\": container with ID starting with 97111fe0a7e8c39c53ed38408be861aa20d439dbde1261fee203af02529409fe not found: ID does not exist" containerID="97111fe0a7e8c39c53ed38408be861aa20d439dbde1261fee203af02529409fe" Jul 24 15:08:46.590259 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:46.590229 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97111fe0a7e8c39c53ed38408be861aa20d439dbde1261fee203af02529409fe"} err="failed to get container status \"97111fe0a7e8c39c53ed38408be861aa20d439dbde1261fee203af02529409fe\": rpc error: code = NotFound desc = could not find container \"97111fe0a7e8c39c53ed38408be861aa20d439dbde1261fee203af02529409fe\": container with ID starting with 97111fe0a7e8c39c53ed38408be861aa20d439dbde1261fee203af02529409fe not found: ID does not exist" Jul 24 15:08:46.592744 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:46.592727 2531 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/88c38c4e-51c6-44fa-8f62-dce6cbfc8569-webhook-cert\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:08:46.592818 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:46.592748 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77fp4\" (UniqueName: \"kubernetes.io/projected/88c38c4e-51c6-44fa-8f62-dce6cbfc8569-kube-api-access-77fp4\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:08:46.603586 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:46.603555 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d8bd4bd76-46dr7"] Jul 24 15:08:46.607701 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:46.607680 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7d8bd4bd76-46dr7"] Jul 24 15:08:48.280181 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:48.280143 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88c38c4e-51c6-44fa-8f62-dce6cbfc8569" path="/var/lib/kubelet/pods/88c38c4e-51c6-44fa-8f62-dce6cbfc8569/volumes" Jul 24 15:08:49.064223 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:08:49.064181 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:10:06.702040 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:10:06.701998 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:10:08.074480 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:10:08.074449 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:10:32.279099 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:10:32.279069 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:10:44.095799 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:10:44.095768 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:10:45.038008 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:10:45.037974 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:10:49.751059 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:10:49.750996 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:10:49.897625 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:10:49.897596 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:10:50.071259 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:10:50.071176 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:10:56.406382 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:10:56.406338 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:10:56.552665 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:10:56.552619 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:10:56.693618 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:10:56.693588 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:10:56.863095 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:10:56.863056 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:11:03.259700 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:11:03.259604 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:11:03.358681 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:11:03.358652 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:11:08.247144 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:11:08.247113 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:11:08.254000 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:11:08.253974 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:11:08.259243 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:11:08.259217 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:11:20.866408 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:11:20.866373 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:11:30.071779 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:11:30.071741 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:11:38.270589 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:11:38.270552 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:11:49.875879 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:11:49.875817 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:12:02.262480 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:12:02.262446 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:12:13.167729 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:12:13.167696 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:12:26.162724 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:12:26.162682 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:12:33.584267 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:12:33.584231 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:12:51.214495 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:12:51.214460 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:12:56.775046 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:12:56.775012 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:13:08.767058 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:13:08.767026 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:13:29.319035 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:13:29.319000 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:13:31.973384 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:13:31.973350 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:13:38.878498 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:13:38.878455 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:13:48.360654 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:13:48.360621 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:13:59.994511 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:13:59.994424 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:14:09.870616 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:14:09.870573 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:14:19.280785 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:14:19.280742 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:14:28.665755 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:14:28.665719 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:14:40.081685 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:14:40.081653 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:15:51.084564 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:15:51.084524 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:16:08.299178 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:16:08.299151 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:16:08.303474 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:16:08.303454 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:16:27.672288 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:16:27.672249 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:16:37.161983 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:16:37.161943 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:16:49.871059 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:16:49.871019 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:17:01.362977 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:17:01.362896 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:17:11.454851 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:17:11.454809 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:17:23.468261 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:17:23.468228 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:17:31.860159 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:17:31.860122 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:17:42.564055 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:17:42.564015 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:17:54.757105 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:17:54.757064 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:18:02.867525 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:18:02.867481 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:18:14.660991 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:18:14.660958 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:18:35.474968 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:18:35.474924 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:18:44.636377 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:18:44.636336 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:18:45.165127 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:18:45.165094 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:18:54.396723 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:18:54.396692 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:18:58.766206 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:18:58.766170 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:19:00.063717 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:19:00.063684 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:19:12.073023 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:19:12.072990 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:19:26.707571 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:19:26.707537 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:19:27.689920 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:19:27.689855 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:19:30.472500 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:19:30.472462 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:19:45.269458 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:19:45.269424 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:19:51.965831 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:19:51.965783 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:20:04.718261 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:20:04.718203 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:20:04.819398 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:20:04.819363 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:20:08.286839 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:20:08.286806 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:20:18.464759 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:20:18.464725 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:20:27.860282 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:20:27.860244 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:20:38.378397 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:20:38.378357 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:20:47.563964 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:20:47.563921 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:20:57.189890 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:20:57.189835 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:21:08.338953 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:21:08.338836 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:21:08.353433 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:21:08.353406 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:21:10.778241 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:21:10.778209 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:21:19.972779 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:21:19.970276 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:21:22.368477 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:21:22.368417 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:21:29.764436 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:21:29.764341 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:21:40.369037 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:21:40.368999 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:21:51.464105 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:21:51.464060 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:21:58.369713 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:21:58.369672 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:22:15.263704 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:22:15.263650 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:22:27.474889 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:22:27.474843 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:22:38.884782 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:22:38.884740 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:22:40.113812 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:22:40.113778 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:22:51.555690 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:22:51.555644 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:23:01.368017 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:01.367922 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:23:01.855702 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:01.855663 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:05.158142 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:05.158103 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:05.255595 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:05.255557 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:05.723906 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:05.723845 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:11.903152 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:11.903110 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:12.007378 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:12.007349 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:12.062992 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:12.062965 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:23:12.160638 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:12.160566 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:12.504422 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:12.504383 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:12.932432 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:12.932396 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:14.702346 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:14.702309 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:14.803708 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:14.803668 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:14.895560 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:14.895357 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:15.198896 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:15.198845 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:15.419946 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:15.419912 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:15.515184 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:15.515101 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:22.270604 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:22.270564 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:23:25.593403 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:25.593366 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:50.978017 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:50.977983 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:52.268613 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:52.268574 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:53.669287 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:53.669251 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:23:55.473292 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:55.473256 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:55.634138 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:55.634102 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:55.891344 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:55.891255 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:56.160631 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:56.160538 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:56.257475 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:56.257436 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:56.374246 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:56.374207 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:56.714926 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:56.714854 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:57.151846 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:57.151740 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:59.062171 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:59.062135 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:59.447073 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:59.447037 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:59.547760 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:59.547705 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:59.759625 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:59.759500 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:23:59.859997 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:23:59.859955 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:24:05.067141 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:24:05.067109 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:24:34.586396 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:24:34.586350 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:24:34.666536 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:24:34.666501 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:24:34.736643 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:24:34.736603 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:24:37.963451 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:24:37.963420 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:24:39.179725 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:24:39.179690 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:24:39.345192 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:24:39.345164 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:24:39.665109 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:24:39.665026 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:24:39.842377 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:24:39.842342 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:24:40.180480 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:24:40.180445 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:24:40.593242 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:24:40.593152 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:24:42.158587 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:24:42.158547 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:24:42.257599 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:24:42.257561 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:24:42.361390 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:24:42.361353 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:24:42.463305 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:24:42.463270 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:24:42.846647 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:24:42.846557 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:24:42.944293 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:24:42.944258 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:24:48.865548 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:24:48.865508 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:25:00.469298 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:25:00.469255 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:25:00.568343 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:25:00.568301 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:25:18.309625 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:25:18.309586 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:25:18.686462 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:25:18.686416 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:25:18.770494 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:25:18.770452 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:25:18.836822 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:25:18.836790 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:25:21.660906 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:25:21.660850 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:25:23.170091 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:25:23.170055 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:25:24.465804 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:25:24.465766 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:25:24.684664 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:25:24.684621 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:25:24.910527 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:25:24.910427 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:25:25.272659 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:25:25.272622 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:25:25.392628 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:25:25.392596 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:25:27.372245 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:25:27.372201 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:25:27.478111 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:25:27.478072 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:25:27.605135 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:25:27.605104 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:25:28.304513 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:25:28.304479 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:25:28.401198 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:25:28.401150 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:25:32.672896 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:25:32.672843 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:26:02.787188 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:02.787130 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:26:02.877819 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:02.877779 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:26:02.952248 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:02.952210 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:26:05.570231 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:05.570188 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:26:07.363787 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:07.363745 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:26:07.529784 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:07.529742 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:26:07.765782 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:07.765743 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:26:08.272572 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:08.272538 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:26:08.363300 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:08.363150 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:26:08.387617 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:08.387590 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:26:08.395893 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:08.395840 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:26:08.466486 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:08.466452 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:26:09.922976 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:09.922942 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:26:16.918438 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:16.918397 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:26:17.057100 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:17.057038 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:26:17.151223 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:17.151188 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:26:17.251402 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:17.251337 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:26:17.650474 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:17.650354 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:26:43.643369 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:43.643331 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:26:50.511614 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:50.511576 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:26:50.682152 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:50.682107 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:26:50.801625 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:26:50.801526 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:06.614827 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:06.614782 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:12.445208 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:12.445163 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:12.551155 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:12.551108 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:12.645451 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:12.645410 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:23.189762 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:23.189720 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:24.010129 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:24.010076 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:30.589363 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:30.589254 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:30.698301 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:30.698253 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:30.797273 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:30.797232 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:30.897378 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:30.897282 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:31.737584 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:31.737542 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:31.831306 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:31.831264 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:31.930384 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:31.930343 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:32.827925 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:32.827878 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:33.566403 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:33.566368 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:34.964533 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:34.964492 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:27:40.196506 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:40.196465 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:40.295718 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:40.295683 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:40.860458 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:40.860423 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:45.164820 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:45.164784 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:27:46.890462 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:46.890422 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:46.978333 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:46.978278 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:47.148404 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:47.148303 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:47.473638 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:47.473597 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:47.860556 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:47.860462 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:50.695464 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:50.695418 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:50.785449 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:50.785408 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:50.985984 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:50.985890 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:51.648417 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:51.648364 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:51.753511 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:51.753473 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:55.167644 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:55.167606 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:27:55.171681 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:55.171654 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:55.831228 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:55.831193 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:55.927307 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:55.927272 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:27:56.269042 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:27:56.269003 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:02.424655 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:02.424611 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:02.526112 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:02.526070 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:12.943927 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:12.943846 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:13.019522 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:13.019479 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:13.088504 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:13.088464 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:15.969908 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:15.969845 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:28:17.880031 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:17.879982 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:18.288632 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:18.288595 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:24.194725 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:24.194684 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:24.294818 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:24.294775 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:24.588936 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:24.588726 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:24.738811 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:24.738776 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:24.875993 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:24.875898 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:24.966731 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:24.966698 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:25.086455 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:25.086415 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:31.246194 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:31.246150 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:31.355247 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:31.355213 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:31.495352 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:31.495321 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:31.605195 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:31.605097 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:37.590450 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:37.590412 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:37.692726 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:37.692692 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:44.432721 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:44.432679 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:44.744055 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:44.743951 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:44.843618 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:44.843580 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:47.916638 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:47.916599 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:48.027130 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:48.027088 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:48.117027 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:48.116998 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:48.232963 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:48.232929 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:48.627832 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:48.627739 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:48.727261 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:48.727222 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:52.437217 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:52.437181 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:28:55.359326 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:55.359276 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:28:56.834348 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:28:56.834287 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:29:06.363177 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:29:06.363086 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:29:13.568439 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:29:13.568397 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:29:36.362318 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:29:36.362266 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:29:37.516597 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:29:37.516548 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:29:37.596658 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:29:37.596621 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:29:37.667716 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:29:37.667675 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:29:41.981471 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:29:41.981422 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:29:45.252423 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:29:45.252386 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:29:46.962704 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:29:46.962659 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:29:47.854854 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:29:47.854813 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:29:50.324165 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:29:50.324129 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:29:59.390030 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:29:59.389994 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:27.061335 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:27.061294 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:27.983788 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:27.983736 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:33.678460 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:33.678415 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:39.319215 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:39.319169 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:39.392938 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:39.392903 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:39.616788 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:39.616682 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:39.885886 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:39.885769 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:41.029263 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:41.029219 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:41.132627 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:41.132589 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:41.228541 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:41.228489 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:41.956664 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:41.956627 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:42.056029 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:42.055992 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:42.157719 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:42.157682 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:43.539265 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:43.539231 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:43.661337 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:43.661297 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:30:49.598057 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:49.598012 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:49.668565 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:49.668516 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:50.321843 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:50.321805 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:51.827808 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:51.827771 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:55.964829 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:55.964779 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:30:56.620202 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:56.620160 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:56.731987 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:56.731944 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:30:56.824591 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:30:56.824552 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:07.385055 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:07.385018 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:07.766092 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:07.766043 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:08.429040 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:08.429008 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:31:08.439434 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:08.439405 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:31:28.731035 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:28.730989 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:35.285123 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:35.285074 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:35.385575 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:35.385529 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:35.644322 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:35.644218 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:35.944524 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:35.944489 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:36.488362 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:36.488311 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:38.738494 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:38.738460 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:38.834843 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:38.834783 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:38.931168 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:38.931132 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:39.406328 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:39.406288 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:39.506081 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:39.506040 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:39.607650 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:39.607618 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:43.252937 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:43.252889 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:43.596420 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:43.596323 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:43.925404 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:43.925359 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:44.236831 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:44.236731 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:44.965972 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:44.965931 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:31:46.816441 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:46.816396 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:47.839485 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:47.839439 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:47.934704 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:47.934662 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:51.286630 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:51.286597 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:31:57.971552 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:57.971512 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:31:58.377085 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:31:58.376985 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:32:07.569528 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:32:07.569439 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:32:24.659148 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:32:24.659112 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:32:25.436019 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:32:25.435986 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:32:28.860227 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:32:28.860189 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:32:42.562860 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:32:42.562818 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:33:06.973196 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:06.973148 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:07.677067 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:07.677021 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:07.809789 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:07.809747 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-92cb39-8kbww"] Jul 24 15:33:07.810331 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:07.810311 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88c38c4e-51c6-44fa-8f62-dce6cbfc8569" containerName="manager" Jul 24 15:33:07.810428 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:07.810334 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="88c38c4e-51c6-44fa-8f62-dce6cbfc8569" containerName="manager" Jul 24 15:33:07.810487 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:07.810468 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="88c38c4e-51c6-44fa-8f62-dce6cbfc8569" containerName="manager" Jul 24 15:33:07.813568 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:07.813545 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-92cb39-8kbww" Jul 24 15:33:07.816350 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:07.816331 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q8grb\"" Jul 24 15:33:07.824261 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:07.824147 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-92cb39-8kbww"] Jul 24 15:33:07.914171 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:07.914126 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba98d91a-6efc-41d8-a184-cc5ce20c0f0f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-92cb39-8kbww\" (UID: \"ba98d91a-6efc-41d8-a184-cc5ce20c0f0f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-92cb39-8kbww" Jul 24 15:33:08.015254 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:08.015153 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba98d91a-6efc-41d8-a184-cc5ce20c0f0f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-92cb39-8kbww\" (UID: \"ba98d91a-6efc-41d8-a184-cc5ce20c0f0f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-92cb39-8kbww" Jul 24 15:33:08.015800 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:08.015777 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba98d91a-6efc-41d8-a184-cc5ce20c0f0f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-92cb39-8kbww\" (UID: \"ba98d91a-6efc-41d8-a184-cc5ce20c0f0f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-92cb39-8kbww" Jul 24 15:33:08.126388 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:08.126350 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-92cb39-8kbww" Jul 24 15:33:08.281139 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:08.281043 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-92cb39-8kbww"] Jul 24 15:33:08.285123 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:33:08.285092 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba98d91a_6efc_41d8_a184_cc5ce20c0f0f.slice/crio-a242d4226717d9ce9e52a2cf85d1252651a7ba1090f74d220809f4f3b52223a3 WatchSource:0}: Error finding container a242d4226717d9ce9e52a2cf85d1252651a7ba1090f74d220809f4f3b52223a3: Status 404 returned error can't find the container with id a242d4226717d9ce9e52a2cf85d1252651a7ba1090f74d220809f4f3b52223a3 Jul 24 15:33:08.286996 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:08.286981 2531 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 15:33:08.424940 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:08.424898 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-92cb39-8kbww" event={"ID":"ba98d91a-6efc-41d8-a184-cc5ce20c0f0f","Type":"ContainerStarted","Data":"a242d4226717d9ce9e52a2cf85d1252651a7ba1090f74d220809f4f3b52223a3"} Jul 24 15:33:11.470333 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:11.470287 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:33:13.462279 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:13.462189 2531 generic.go:356] "Generic (PLEG): container finished" podID="ba98d91a-6efc-41d8-a184-cc5ce20c0f0f" containerID="c926ad26e9524ba53e3e6d1267c9287875cdf98b6bc1a74be5bbccb75efa796c" exitCode=0 Jul 24 15:33:13.462687 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:13.462274 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-92cb39-8kbww" event={"ID":"ba98d91a-6efc-41d8-a184-cc5ce20c0f0f","Type":"ContainerDied","Data":"c926ad26e9524ba53e3e6d1267c9287875cdf98b6bc1a74be5bbccb75efa796c"} Jul 24 15:33:14.616930 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:14.616894 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-92cb39-8kbww" Jul 24 15:33:14.680076 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:14.680037 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba98d91a-6efc-41d8-a184-cc5ce20c0f0f-maas-api-service-ca\") pod \"ba98d91a-6efc-41d8-a184-cc5ce20c0f0f\" (UID: \"ba98d91a-6efc-41d8-a184-cc5ce20c0f0f\") " Jul 24 15:33:14.680417 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:14.680394 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba98d91a-6efc-41d8-a184-cc5ce20c0f0f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ba98d91a-6efc-41d8-a184-cc5ce20c0f0f" (UID: "ba98d91a-6efc-41d8-a184-cc5ce20c0f0f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:33:14.781264 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:14.781167 2531 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba98d91a-6efc-41d8-a184-cc5ce20c0f0f-maas-api-service-ca\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:33:15.476663 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:15.476632 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-92cb39-8kbww" Jul 24 15:33:15.476843 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:15.476627 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-92cb39-8kbww" event={"ID":"ba98d91a-6efc-41d8-a184-cc5ce20c0f0f","Type":"ContainerDied","Data":"a242d4226717d9ce9e52a2cf85d1252651a7ba1090f74d220809f4f3b52223a3"} Jul 24 15:33:15.476843 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:15.476760 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a242d4226717d9ce9e52a2cf85d1252651a7ba1090f74d220809f4f3b52223a3" Jul 24 15:33:18.442000 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:18.441955 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:18.543136 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:18.543100 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:18.642648 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:18.642609 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:22.886938 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:22.886904 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:22.916519 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:22.916484 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-92cb39-8kbww"] Jul 24 15:33:22.919170 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:22.919147 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-92cb39-8kbww"] Jul 24 15:33:23.297622 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:23.297584 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:24.064338 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:24.064289 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:33:24.278962 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:24.278926 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba98d91a-6efc-41d8-a184-cc5ce20c0f0f" path="/var/lib/kubelet/pods/ba98d91a-6efc-41d8-a184-cc5ce20c0f0f/volumes" Jul 24 15:33:29.486711 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:29.486615 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:29.550225 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:29.550188 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:29.773205 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:29.773103 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:30.028143 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:30.028046 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:30.147358 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:30.147314 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dde49b-4n9z8"] Jul 24 15:33:30.147778 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:30.147766 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba98d91a-6efc-41d8-a184-cc5ce20c0f0f" containerName="revoke-keys" Jul 24 15:33:30.147824 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:30.147780 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba98d91a-6efc-41d8-a184-cc5ce20c0f0f" containerName="revoke-keys" Jul 24 15:33:30.147899 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:30.147858 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba98d91a-6efc-41d8-a184-cc5ce20c0f0f" containerName="revoke-keys" Jul 24 15:33:30.152183 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:30.152158 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dde49b-4n9z8" Jul 24 15:33:30.154683 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:30.154661 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q8grb\"" Jul 24 15:33:30.159320 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:30.159289 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dde49b-4n9z8"] Jul 24 15:33:30.224063 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:30.224020 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e74a214c-3cc9-4f05-ba1d-c295b0e1e771-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-dde49b-4n9z8\" (UID: \"e74a214c-3cc9-4f05-ba1d-c295b0e1e771\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dde49b-4n9z8" Jul 24 15:33:30.324635 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:30.324533 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e74a214c-3cc9-4f05-ba1d-c295b0e1e771-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-dde49b-4n9z8\" (UID: \"e74a214c-3cc9-4f05-ba1d-c295b0e1e771\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dde49b-4n9z8" Jul 24 15:33:30.325283 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:30.325260 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e74a214c-3cc9-4f05-ba1d-c295b0e1e771-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-dde49b-4n9z8\" (UID: \"e74a214c-3cc9-4f05-ba1d-c295b0e1e771\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dde49b-4n9z8" Jul 24 15:33:30.465025 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:30.464982 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dde49b-4n9z8" Jul 24 15:33:30.616706 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:30.616680 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dde49b-4n9z8"] Jul 24 15:33:30.618557 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:33:30.618531 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode74a214c_3cc9_4f05_ba1d_c295b0e1e771.slice/crio-ee2dd9dfdc33d562d3d2b309fd69228139b760cf0b71b06b6cbb93d3613471c0 WatchSource:0}: Error finding container ee2dd9dfdc33d562d3d2b309fd69228139b760cf0b71b06b6cbb93d3613471c0: Status 404 returned error can't find the container with id ee2dd9dfdc33d562d3d2b309fd69228139b760cf0b71b06b6cbb93d3613471c0 Jul 24 15:33:31.578503 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:31.578410 2531 generic.go:356] "Generic (PLEG): container finished" podID="e74a214c-3cc9-4f05-ba1d-c295b0e1e771" containerID="21e89ee14705900c480a3f2c160da8b94ed68ba4353ba237473d1eb3b605be01" exitCode=0 Jul 24 15:33:31.578652 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:31.578509 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dde49b-4n9z8" event={"ID":"e74a214c-3cc9-4f05-ba1d-c295b0e1e771","Type":"ContainerDied","Data":"21e89ee14705900c480a3f2c160da8b94ed68ba4353ba237473d1eb3b605be01"} Jul 24 15:33:31.578652 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:31.578569 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dde49b-4n9z8" event={"ID":"e74a214c-3cc9-4f05-ba1d-c295b0e1e771","Type":"ContainerStarted","Data":"ee2dd9dfdc33d562d3d2b309fd69228139b760cf0b71b06b6cbb93d3613471c0"} Jul 24 15:33:32.731856 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:32.731824 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dde49b-4n9z8" Jul 24 15:33:32.848172 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:32.848086 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e74a214c-3cc9-4f05-ba1d-c295b0e1e771-maas-api-service-ca\") pod \"e74a214c-3cc9-4f05-ba1d-c295b0e1e771\" (UID: \"e74a214c-3cc9-4f05-ba1d-c295b0e1e771\") " Jul 24 15:33:32.848456 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:32.848434 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e74a214c-3cc9-4f05-ba1d-c295b0e1e771-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e74a214c-3cc9-4f05-ba1d-c295b0e1e771" (UID: "e74a214c-3cc9-4f05-ba1d-c295b0e1e771"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:33:32.949521 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:32.949484 2531 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e74a214c-3cc9-4f05-ba1d-c295b0e1e771-maas-api-service-ca\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:33:33.593368 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:33.593340 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dde49b-4n9z8" Jul 24 15:33:33.593549 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:33.593336 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dde49b-4n9z8" event={"ID":"e74a214c-3cc9-4f05-ba1d-c295b0e1e771","Type":"ContainerDied","Data":"ee2dd9dfdc33d562d3d2b309fd69228139b760cf0b71b06b6cbb93d3613471c0"} Jul 24 15:33:33.593549 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:33.593474 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee2dd9dfdc33d562d3d2b309fd69228139b760cf0b71b06b6cbb93d3613471c0" Jul 24 15:33:40.751824 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:40.751790 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:40.832816 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:40.832771 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:40.892313 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:40.892271 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:43.659657 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:43.659608 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:33:45.232267 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:45.232226 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dde49b-4n9z8"] Jul 24 15:33:45.235165 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:45.235094 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-dde49b-4n9z8"] Jul 24 15:33:45.237421 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:45.237389 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:45.716389 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:45.716353 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:46.277905 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:46.277827 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e74a214c-3cc9-4f05-ba1d-c295b0e1e771" path="/var/lib/kubelet/pods/e74a214c-3cc9-4f05-ba1d-c295b0e1e771/volumes" Jul 24 15:33:51.513263 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:51.513223 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:51.613456 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:51.613393 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:51.816673 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:51.816583 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:52.065629 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:52.065594 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:52.448273 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:52.448220 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:52.790959 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:52.790843 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:53.128929 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:53.128786 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:55.444267 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:55.444223 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:56.665171 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:56.665139 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:33:58.671405 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:58.671359 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:33:59.958439 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:33:59.958372 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:00.440351 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:00.440311 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:00.789704 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:00.789612 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:00.959007 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:00.958967 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:03.510823 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:03.510778 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:03.599148 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:03.599111 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:03.701066 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:03.701015 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:03.831998 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:03.831860 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:04.461362 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:04.461329 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:04.553846 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:04.553800 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:07.982469 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:07.982420 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:14.565686 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:14.565643 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:34:16.170026 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:16.169981 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f97fff-nc49h"] Jul 24 15:34:16.170456 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:16.170442 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e74a214c-3cc9-4f05-ba1d-c295b0e1e771" containerName="revoke-keys" Jul 24 15:34:16.170502 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:16.170457 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="e74a214c-3cc9-4f05-ba1d-c295b0e1e771" containerName="revoke-keys" Jul 24 15:34:16.170552 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:16.170543 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="e74a214c-3cc9-4f05-ba1d-c295b0e1e771" containerName="revoke-keys" Jul 24 15:34:16.173833 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:16.173807 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f97fff-nc49h" Jul 24 15:34:16.176561 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:16.176527 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q8grb\"" Jul 24 15:34:16.183763 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:16.183731 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f97fff-nc49h"] Jul 24 15:34:16.282286 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:16.282239 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef591fec-b3dd-44cf-81fb-b74776b29c0b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f97fff-nc49h\" (UID: \"ef591fec-b3dd-44cf-81fb-b74776b29c0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f97fff-nc49h" Jul 24 15:34:16.383837 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:16.383778 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef591fec-b3dd-44cf-81fb-b74776b29c0b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f97fff-nc49h\" (UID: \"ef591fec-b3dd-44cf-81fb-b74776b29c0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f97fff-nc49h" Jul 24 15:34:16.384448 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:16.384427 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef591fec-b3dd-44cf-81fb-b74776b29c0b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-f97fff-nc49h\" (UID: \"ef591fec-b3dd-44cf-81fb-b74776b29c0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f97fff-nc49h" Jul 24 15:34:16.487084 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:16.487050 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f97fff-nc49h" Jul 24 15:34:16.868694 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:16.862338 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f97fff-nc49h"] Jul 24 15:34:17.870234 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:17.870197 2531 generic.go:356] "Generic (PLEG): container finished" podID="ef591fec-b3dd-44cf-81fb-b74776b29c0b" containerID="4d40e3a4d7aa357d73bb8e5b0785d1f431dfb8d3968819d19d8c280c1015a70d" exitCode=0 Jul 24 15:34:17.870687 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:17.870283 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f97fff-nc49h" event={"ID":"ef591fec-b3dd-44cf-81fb-b74776b29c0b","Type":"ContainerDied","Data":"4d40e3a4d7aa357d73bb8e5b0785d1f431dfb8d3968819d19d8c280c1015a70d"} Jul 24 15:34:17.870687 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:17.870338 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f97fff-nc49h" event={"ID":"ef591fec-b3dd-44cf-81fb-b74776b29c0b","Type":"ContainerStarted","Data":"5b320dfbd117e6630eec95148a7cac9a8ee704f3b3c28f4bf6d733f73d4a8819"} Jul 24 15:34:19.028033 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:19.028004 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f97fff-nc49h" Jul 24 15:34:19.109526 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:19.109482 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef591fec-b3dd-44cf-81fb-b74776b29c0b-maas-api-service-ca\") pod \"ef591fec-b3dd-44cf-81fb-b74776b29c0b\" (UID: \"ef591fec-b3dd-44cf-81fb-b74776b29c0b\") " Jul 24 15:34:19.109899 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:19.109843 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef591fec-b3dd-44cf-81fb-b74776b29c0b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ef591fec-b3dd-44cf-81fb-b74776b29c0b" (UID: "ef591fec-b3dd-44cf-81fb-b74776b29c0b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:34:19.211011 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:19.210971 2531 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef591fec-b3dd-44cf-81fb-b74776b29c0b-maas-api-service-ca\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:34:19.883907 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:19.883856 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f97fff-nc49h" Jul 24 15:34:19.883907 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:19.883903 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f97fff-nc49h" event={"ID":"ef591fec-b3dd-44cf-81fb-b74776b29c0b","Type":"ContainerDied","Data":"5b320dfbd117e6630eec95148a7cac9a8ee704f3b3c28f4bf6d733f73d4a8819"} Jul 24 15:34:19.884183 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:19.883953 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b320dfbd117e6630eec95148a7cac9a8ee704f3b3c28f4bf6d733f73d4a8819" Jul 24 15:34:24.665418 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:24.665378 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:24.779424 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:24.779391 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:31.887452 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:31.887406 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:31.983940 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:31.983902 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:32.092801 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:32.092760 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:34.465670 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:34.465630 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:34:36.258954 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:36.258913 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f97fff-nc49h"] Jul 24 15:34:36.261307 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:36.261265 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:36.262653 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:36.262631 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-f97fff-nc49h"] Jul 24 15:34:36.277847 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:36.277813 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef591fec-b3dd-44cf-81fb-b74776b29c0b" path="/var/lib/kubelet/pods/ef591fec-b3dd-44cf-81fb-b74776b29c0b/volumes" Jul 24 15:34:36.702377 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:36.702321 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:42.847500 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:42.847465 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:42.952767 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:42.952728 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:43.189994 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:43.189953 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:43.404551 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:43.404520 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:43.522182 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:43.522080 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-03bf4f-j56vv"] Jul 24 15:34:43.522583 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:43.522567 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef591fec-b3dd-44cf-81fb-b74776b29c0b" containerName="revoke-keys" Jul 24 15:34:43.522634 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:43.522587 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef591fec-b3dd-44cf-81fb-b74776b29c0b" containerName="revoke-keys" Jul 24 15:34:43.522695 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:43.522685 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef591fec-b3dd-44cf-81fb-b74776b29c0b" containerName="revoke-keys" Jul 24 15:34:43.526885 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:43.526837 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-03bf4f-j56vv" Jul 24 15:34:43.530807 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:43.529802 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q8grb\"" Jul 24 15:34:43.535723 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:43.535696 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-03bf4f-j56vv"] Jul 24 15:34:43.551412 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:43.551361 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f231cf55-b1b9-41ae-ac65-1829ab5088af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-03bf4f-j56vv\" (UID: \"f231cf55-b1b9-41ae-ac65-1829ab5088af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-03bf4f-j56vv" Jul 24 15:34:43.652076 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:43.652040 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f231cf55-b1b9-41ae-ac65-1829ab5088af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-03bf4f-j56vv\" (UID: \"f231cf55-b1b9-41ae-ac65-1829ab5088af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-03bf4f-j56vv" Jul 24 15:34:43.652822 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:43.652789 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f231cf55-b1b9-41ae-ac65-1829ab5088af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-03bf4f-j56vv\" (UID: \"f231cf55-b1b9-41ae-ac65-1829ab5088af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-03bf4f-j56vv" Jul 24 15:34:43.841512 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:43.841414 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-03bf4f-j56vv" Jul 24 15:34:43.993349 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:43.993314 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-03bf4f-j56vv"] Jul 24 15:34:43.994795 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:34:43.994766 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf231cf55_b1b9_41ae_ac65_1829ab5088af.slice/crio-c6c725522df86a6c2a3f35bae85cef284597ba9ab6f8937d420f66e5f29494fa WatchSource:0}: Error finding container c6c725522df86a6c2a3f35bae85cef284597ba9ab6f8937d420f66e5f29494fa: Status 404 returned error can't find the container with id c6c725522df86a6c2a3f35bae85cef284597ba9ab6f8937d420f66e5f29494fa Jul 24 15:34:44.032395 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:44.032356 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-03bf4f-j56vv" event={"ID":"f231cf55-b1b9-41ae-ac65-1829ab5088af","Type":"ContainerStarted","Data":"c6c725522df86a6c2a3f35bae85cef284597ba9ab6f8937d420f66e5f29494fa"} Jul 24 15:34:45.042509 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:45.042404 2531 generic.go:356] "Generic (PLEG): container finished" podID="f231cf55-b1b9-41ae-ac65-1829ab5088af" containerID="39eae45741f287275d385b33efbd8fbd01f021e19406bee36483e45384031620" exitCode=0 Jul 24 15:34:45.042509 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:45.042496 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-03bf4f-j56vv" event={"ID":"f231cf55-b1b9-41ae-ac65-1829ab5088af","Type":"ContainerDied","Data":"39eae45741f287275d385b33efbd8fbd01f021e19406bee36483e45384031620"} Jul 24 15:34:46.195041 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:46.195014 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-03bf4f-j56vv" Jul 24 15:34:46.279523 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:46.279491 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f231cf55-b1b9-41ae-ac65-1829ab5088af-maas-api-service-ca\") pod \"f231cf55-b1b9-41ae-ac65-1829ab5088af\" (UID: \"f231cf55-b1b9-41ae-ac65-1829ab5088af\") " Jul 24 15:34:46.279901 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:46.279852 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f231cf55-b1b9-41ae-ac65-1829ab5088af-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f231cf55-b1b9-41ae-ac65-1829ab5088af" (UID: "f231cf55-b1b9-41ae-ac65-1829ab5088af"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:34:46.381182 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:46.381073 2531 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f231cf55-b1b9-41ae-ac65-1829ab5088af-maas-api-service-ca\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:34:47.055384 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:47.055348 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-03bf4f-j56vv" Jul 24 15:34:47.055560 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:47.055389 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-03bf4f-j56vv" event={"ID":"f231cf55-b1b9-41ae-ac65-1829ab5088af","Type":"ContainerDied","Data":"c6c725522df86a6c2a3f35bae85cef284597ba9ab6f8937d420f66e5f29494fa"} Jul 24 15:34:47.055560 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:47.055441 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c6c725522df86a6c2a3f35bae85cef284597ba9ab6f8937d420f66e5f29494fa" Jul 24 15:34:59.016193 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:59.016160 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:59.086722 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:59.086681 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:34:59.188301 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:34:59.188265 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:03.605097 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:03.605062 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:03.612580 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:03.612491 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-03bf4f-j56vv"] Jul 24 15:35:03.615144 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:03.615122 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-03bf4f-j56vv"] Jul 24 15:35:03.775238 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:03.775202 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:35:04.004861 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:04.004821 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:04.278031 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:04.277951 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f231cf55-b1b9-41ae-ac65-1829ab5088af" path="/var/lib/kubelet/pods/f231cf55-b1b9-41ae-ac65-1829ab5088af/volumes" Jul 24 15:35:09.842853 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:09.842808 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:11.223823 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:11.223787 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:12.394822 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:12.394790 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:12.453437 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:12.453405 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:13.806308 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:13.806272 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:13.904637 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:13.904601 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:13.986067 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:13.986027 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:14.318974 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:14.318941 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:14.515712 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:14.515645 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:14.611137 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:14.611044 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:18.338930 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:18.338894 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:18.440545 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:18.440514 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:18.682017 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:18.681972 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:19.007008 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:19.006843 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:19.309372 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:19.309270 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:21.889395 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:21.889346 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:21.970780 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:21.970744 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:21.972829 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:21.972799 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:35:22.052667 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:22.052629 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:22.222674 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:22.222634 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:22.634823 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:22.634694 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:22.735823 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:22.735787 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:26.375843 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:26.375809 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:33.281740 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:33.281701 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:35:42.778365 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:42.778324 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:35:44.863360 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:35:44.863317 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:36:05.991335 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:36:05.991292 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:36:08.480923 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:36:08.480749 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:36:08.492679 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:36:08.492650 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:36:10.062606 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:36:10.062570 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:36:33.566294 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:36:33.566249 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:36:52.795751 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:36:52.795714 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:37:05.487753 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:37:05.487708 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:37:23.463896 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:37:23.463826 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:37:38.666899 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:37:38.666843 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:37:54.951572 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:37:54.951528 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:38:14.457020 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:38:14.456932 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:38:29.972682 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:38:29.972638 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:38:30.968121 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:38:30.968078 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:38:35.152512 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:38:35.152474 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:38:42.079612 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:38:42.079576 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:38:52.758856 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:38:52.758806 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:39:07.530836 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:39:07.530798 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:39:18.433629 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:39:18.433588 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:39:23.601857 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:39:23.601796 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:39:27.363900 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:39:27.363845 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:39:36.792376 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:39:36.792340 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:39:38.725197 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:39:38.725157 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:39:43.357707 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:39:43.357661 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:39:58.149568 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:39:58.149522 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:01.764992 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:40:01.764944 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:40:07.439646 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:40:07.439605 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:08.570434 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:40:08.570396 2531 scope.go:117] "RemoveContainer" containerID="21e89ee14705900c480a3f2c160da8b94ed68ba4353ba237473d1eb3b605be01" Jul 24 15:40:08.580904 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:40:08.580843 2531 scope.go:117] "RemoveContainer" containerID="c926ad26e9524ba53e3e6d1267c9287875cdf98b6bc1a74be5bbccb75efa796c" Jul 24 15:40:08.827969 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:40:08.827885 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:11.390642 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:40:11.390613 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:19.359583 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:40:19.359534 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:40:48.080740 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:40:48.080707 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:40:51.456400 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:40:51.456357 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:41:06.316918 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:06.316846 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:06.377164 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:06.377132 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:08.523459 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:08.523345 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:41:08.536059 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:08.536034 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:41:08.613356 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:08.613322 2531 scope.go:117] "RemoveContainer" containerID="39eae45741f287275d385b33efbd8fbd01f021e19406bee36483e45384031620" Jul 24 15:41:08.622971 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:08.622947 2531 scope.go:117] "RemoveContainer" containerID="4d40e3a4d7aa357d73bb8e5b0785d1f431dfb8d3968819d19d8c280c1015a70d" Jul 24 15:41:14.538135 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:14.538098 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:14.643125 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:14.643090 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:15.758711 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:15.758668 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:41:30.862943 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:30.862892 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:41:38.833243 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:38.833204 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:39.752959 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:39.752921 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:40.743969 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:40.743929 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7491-140675006847280"] Jul 24 15:41:40.744376 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:40.744363 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f231cf55-b1b9-41ae-ac65-1829ab5088af" containerName="revoke-keys" Jul 24 15:41:40.744422 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:40.744377 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="f231cf55-b1b9-41ae-ac65-1829ab5088af" containerName="revoke-keys" Jul 24 15:41:40.744472 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:40.744462 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="f231cf55-b1b9-41ae-ac65-1829ab5088af" containerName="revoke-keys" Jul 24 15:41:40.747516 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:40.747498 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675006847280" Jul 24 15:41:40.750384 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:40.749950 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-rqfdc\"" Jul 24 15:41:40.753789 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:40.753760 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140675006847280"] Jul 24 15:41:40.809473 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:40.809440 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnqps\" (UniqueName: \"kubernetes.io/projected/1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9-kube-api-access-bnqps\") pod \"test-curl-7491-140675006847280\" (UID: \"1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9\") " pod="opendatahub/test-curl-7491-140675006847280" Jul 24 15:41:40.909994 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:40.909951 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnqps\" (UniqueName: \"kubernetes.io/projected/1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9-kube-api-access-bnqps\") pod \"test-curl-7491-140675006847280\" (UID: \"1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9\") " pod="opendatahub/test-curl-7491-140675006847280" Jul 24 15:41:40.917645 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:40.917617 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnqps\" (UniqueName: \"kubernetes.io/projected/1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9-kube-api-access-bnqps\") pod \"test-curl-7491-140675006847280\" (UID: \"1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9\") " pod="opendatahub/test-curl-7491-140675006847280" Jul 24 15:41:41.059730 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:41.059634 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675006847280" Jul 24 15:41:41.204531 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:41.204497 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140675006847280"] Jul 24 15:41:41.206308 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:41:41.206281 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1755779e_1fd6_4ff4_96ab_5b31d3bd4eb9.slice/crio-5571c7a065cd09067db995683fe0945ffae962f95c171bb0394b6e1e08a84e26 WatchSource:0}: Error finding container 5571c7a065cd09067db995683fe0945ffae962f95c171bb0394b6e1e08a84e26: Status 404 returned error can't find the container with id 5571c7a065cd09067db995683fe0945ffae962f95c171bb0394b6e1e08a84e26 Jul 24 15:41:41.207605 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:41.207588 2531 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 15:41:41.700624 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:41.700579 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140675006847280" event={"ID":"1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9","Type":"ContainerStarted","Data":"5571c7a065cd09067db995683fe0945ffae962f95c171bb0394b6e1e08a84e26"} Jul 24 15:41:42.708225 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:42.708193 2531 generic.go:356] "Generic (PLEG): container finished" podID="1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9" containerID="c469db8746a70beca4ab96de4383fcc4e35e29d95f06218c3aad2b6a804f76ee" exitCode=0 Jul 24 15:41:42.708682 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:42.708277 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140675006847280" event={"ID":"1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9","Type":"ContainerDied","Data":"c469db8746a70beca4ab96de4383fcc4e35e29d95f06218c3aad2b6a804f76ee"} Jul 24 15:41:43.859501 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:43.859476 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675006847280" Jul 24 15:41:43.885624 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:43.885590 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7491-140675006847280_1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9/test-curl-7491-140675006847280/0.log" Jul 24 15:41:43.912298 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:43.912242 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7491-140675006847280"] Jul 24 15:41:43.916232 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:43.916196 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7491-140675006847280"] Jul 24 15:41:43.939846 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:43.939813 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnqps\" (UniqueName: \"kubernetes.io/projected/1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9-kube-api-access-bnqps\") pod \"1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9\" (UID: \"1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9\") " Jul 24 15:41:43.942210 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:43.942142 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9-kube-api-access-bnqps" (OuterVolumeSpecName: "kube-api-access-bnqps") pod "1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9" (UID: "1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9"). InnerVolumeSpecName "kube-api-access-bnqps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:41:44.037135 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:44.037041 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7491-140675006847408"] Jul 24 15:41:44.037532 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:44.037515 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9" containerName="test-curl-7491-140675006847280" Jul 24 15:41:44.037577 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:44.037533 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9" containerName="test-curl-7491-140675006847280" Jul 24 15:41:44.037612 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:44.037605 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9" containerName="test-curl-7491-140675006847280" Jul 24 15:41:44.040786 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:44.040754 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnqps\" (UniqueName: \"kubernetes.io/projected/1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9-kube-api-access-bnqps\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:41:44.040909 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:44.040854 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675006847408" Jul 24 15:41:44.047254 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:44.047221 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140675006847408"] Jul 24 15:41:44.141828 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:44.141778 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbv4z\" (UniqueName: \"kubernetes.io/projected/5d1b60c2-7095-459f-ba61-6363117fef7a-kube-api-access-bbv4z\") pod \"test-curl-7491-140675006847408\" (UID: \"5d1b60c2-7095-459f-ba61-6363117fef7a\") " pod="opendatahub/test-curl-7491-140675006847408" Jul 24 15:41:44.243264 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:44.243215 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbv4z\" (UniqueName: \"kubernetes.io/projected/5d1b60c2-7095-459f-ba61-6363117fef7a-kube-api-access-bbv4z\") pod \"test-curl-7491-140675006847408\" (UID: \"5d1b60c2-7095-459f-ba61-6363117fef7a\") " pod="opendatahub/test-curl-7491-140675006847408" Jul 24 15:41:44.251989 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:44.251960 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbv4z\" (UniqueName: \"kubernetes.io/projected/5d1b60c2-7095-459f-ba61-6363117fef7a-kube-api-access-bbv4z\") pod \"test-curl-7491-140675006847408\" (UID: \"5d1b60c2-7095-459f-ba61-6363117fef7a\") " pod="opendatahub/test-curl-7491-140675006847408" Jul 24 15:41:44.278313 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:44.278271 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9" path="/var/lib/kubelet/pods/1755779e-1fd6-4ff4-96ab-5b31d3bd4eb9/volumes" Jul 24 15:41:44.353025 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:44.352935 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675006847408" Jul 24 15:41:44.501209 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:44.501173 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140675006847408"] Jul 24 15:41:44.502451 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:41:44.502423 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d1b60c2_7095_459f_ba61_6363117fef7a.slice/crio-64b7396496d2a3ea181d7977337be3574d9a0453bbec2b242d7941e3761d837b WatchSource:0}: Error finding container 64b7396496d2a3ea181d7977337be3574d9a0453bbec2b242d7941e3761d837b: Status 404 returned error can't find the container with id 64b7396496d2a3ea181d7977337be3574d9a0453bbec2b242d7941e3761d837b Jul 24 15:41:44.722907 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:44.722832 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140675006847408" event={"ID":"5d1b60c2-7095-459f-ba61-6363117fef7a","Type":"ContainerStarted","Data":"64b7396496d2a3ea181d7977337be3574d9a0453bbec2b242d7941e3761d837b"} Jul 24 15:41:44.724116 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:44.724100 2531 scope.go:117] "RemoveContainer" containerID="c469db8746a70beca4ab96de4383fcc4e35e29d95f06218c3aad2b6a804f76ee" Jul 24 15:41:44.724186 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:44.724121 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675006847280" Jul 24 15:41:45.733492 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:45.733459 2531 generic.go:356] "Generic (PLEG): container finished" podID="5d1b60c2-7095-459f-ba61-6363117fef7a" containerID="8fce821a6cc609b385682ad97e66c6654fe76d051c6efb22a0a4e56cf9737e67" exitCode=0 Jul 24 15:41:45.734038 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:45.733520 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140675006847408" event={"ID":"5d1b60c2-7095-459f-ba61-6363117fef7a","Type":"ContainerDied","Data":"8fce821a6cc609b385682ad97e66c6654fe76d051c6efb22a0a4e56cf9737e67"} Jul 24 15:41:45.764059 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:45.764019 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:41:46.882136 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:46.882113 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675006847408" Jul 24 15:41:46.900673 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:46.900641 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7491-140675006847408_5d1b60c2-7095-459f-ba61-6363117fef7a/test-curl-7491-140675006847408/0.log" Jul 24 15:41:46.925906 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:46.925833 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7491-140675006847408"] Jul 24 15:41:46.936540 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:46.936491 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7491-140675006847408"] Jul 24 15:41:46.971071 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:46.971041 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbv4z\" (UniqueName: \"kubernetes.io/projected/5d1b60c2-7095-459f-ba61-6363117fef7a-kube-api-access-bbv4z\") pod \"5d1b60c2-7095-459f-ba61-6363117fef7a\" (UID: \"5d1b60c2-7095-459f-ba61-6363117fef7a\") " Jul 24 15:41:46.973363 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:46.973323 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d1b60c2-7095-459f-ba61-6363117fef7a-kube-api-access-bbv4z" (OuterVolumeSpecName: "kube-api-access-bbv4z") pod "5d1b60c2-7095-459f-ba61-6363117fef7a" (UID: "5d1b60c2-7095-459f-ba61-6363117fef7a"). InnerVolumeSpecName "kube-api-access-bbv4z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:41:47.072107 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:47.072009 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbv4z\" (UniqueName: \"kubernetes.io/projected/5d1b60c2-7095-459f-ba61-6363117fef7a-kube-api-access-bbv4z\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:41:47.098017 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:47.097971 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7491-140675008579312"] Jul 24 15:41:47.098465 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:47.098450 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d1b60c2-7095-459f-ba61-6363117fef7a" containerName="test-curl-7491-140675006847408" Jul 24 15:41:47.098525 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:47.098467 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d1b60c2-7095-459f-ba61-6363117fef7a" containerName="test-curl-7491-140675006847408" Jul 24 15:41:47.098564 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:47.098551 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d1b60c2-7095-459f-ba61-6363117fef7a" containerName="test-curl-7491-140675006847408" Jul 24 15:41:47.102069 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:47.102047 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675008579312" Jul 24 15:41:47.107392 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:47.107359 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140675008579312"] Jul 24 15:41:47.172780 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:47.172742 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp886\" (UniqueName: \"kubernetes.io/projected/6acb9fe8-d357-4e1f-9b42-acd8907d159c-kube-api-access-dp886\") pod \"test-curl-7491-140675008579312\" (UID: \"6acb9fe8-d357-4e1f-9b42-acd8907d159c\") " pod="opendatahub/test-curl-7491-140675008579312" Jul 24 15:41:47.273747 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:47.273710 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp886\" (UniqueName: \"kubernetes.io/projected/6acb9fe8-d357-4e1f-9b42-acd8907d159c-kube-api-access-dp886\") pod \"test-curl-7491-140675008579312\" (UID: \"6acb9fe8-d357-4e1f-9b42-acd8907d159c\") " pod="opendatahub/test-curl-7491-140675008579312" Jul 24 15:41:47.283454 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:47.283406 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp886\" (UniqueName: \"kubernetes.io/projected/6acb9fe8-d357-4e1f-9b42-acd8907d159c-kube-api-access-dp886\") pod \"test-curl-7491-140675008579312\" (UID: \"6acb9fe8-d357-4e1f-9b42-acd8907d159c\") " pod="opendatahub/test-curl-7491-140675008579312" Jul 24 15:41:47.414642 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:47.414554 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675008579312" Jul 24 15:41:47.560230 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:47.560204 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140675008579312"] Jul 24 15:41:47.561321 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:41:47.561297 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6acb9fe8_d357_4e1f_9b42_acd8907d159c.slice/crio-2af4a4866f89d170a8786b8ca5477172414e4bb56a5e51e02e8cf2844dfdaf7e WatchSource:0}: Error finding container 2af4a4866f89d170a8786b8ca5477172414e4bb56a5e51e02e8cf2844dfdaf7e: Status 404 returned error can't find the container with id 2af4a4866f89d170a8786b8ca5477172414e4bb56a5e51e02e8cf2844dfdaf7e Jul 24 15:41:47.747003 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:47.746959 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64b7396496d2a3ea181d7977337be3574d9a0453bbec2b242d7941e3761d837b" Jul 24 15:41:47.747216 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:47.747010 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675006847408" Jul 24 15:41:47.748358 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:47.748318 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140675008579312" event={"ID":"6acb9fe8-d357-4e1f-9b42-acd8907d159c","Type":"ContainerStarted","Data":"2af4a4866f89d170a8786b8ca5477172414e4bb56a5e51e02e8cf2844dfdaf7e"} Jul 24 15:41:47.757963 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:41:47.757931 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7491-140675006847408\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="5d1b60c2-7095-459f-ba61-6363117fef7a" pod="opendatahub/test-curl-7491-140675006847408" Jul 24 15:41:48.276488 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:41:48.276443 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7491-140675006847408\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="5d1b60c2-7095-459f-ba61-6363117fef7a" pod="opendatahub/test-curl-7491-140675006847408" Jul 24 15:41:48.281082 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:48.281047 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d1b60c2-7095-459f-ba61-6363117fef7a" path="/var/lib/kubelet/pods/5d1b60c2-7095-459f-ba61-6363117fef7a/volumes" Jul 24 15:41:48.756307 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:48.756275 2531 generic.go:356] "Generic (PLEG): container finished" podID="6acb9fe8-d357-4e1f-9b42-acd8907d159c" containerID="38af35fff13b89db17aae506b04d7b69a0dfd74a01d84dcfd40856d8ff78e3df" exitCode=0 Jul 24 15:41:48.756496 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:48.756358 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140675008579312" event={"ID":"6acb9fe8-d357-4e1f-9b42-acd8907d159c","Type":"ContainerDied","Data":"38af35fff13b89db17aae506b04d7b69a0dfd74a01d84dcfd40856d8ff78e3df"} Jul 24 15:41:49.907927 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:49.907903 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675008579312" Jul 24 15:41:49.927551 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:49.927520 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7491-140675008579312_6acb9fe8-d357-4e1f-9b42-acd8907d159c/test-curl-7491-140675008579312/0.log" Jul 24 15:41:49.952948 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:49.952912 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7491-140675008579312"] Jul 24 15:41:49.956097 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:49.956069 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7491-140675008579312"] Jul 24 15:41:50.002401 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:50.002366 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dp886\" (UniqueName: \"kubernetes.io/projected/6acb9fe8-d357-4e1f-9b42-acd8907d159c-kube-api-access-dp886\") pod \"6acb9fe8-d357-4e1f-9b42-acd8907d159c\" (UID: \"6acb9fe8-d357-4e1f-9b42-acd8907d159c\") " Jul 24 15:41:50.004664 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:50.004637 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6acb9fe8-d357-4e1f-9b42-acd8907d159c-kube-api-access-dp886" (OuterVolumeSpecName: "kube-api-access-dp886") pod "6acb9fe8-d357-4e1f-9b42-acd8907d159c" (UID: "6acb9fe8-d357-4e1f-9b42-acd8907d159c"). InnerVolumeSpecName "kube-api-access-dp886". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:41:50.082736 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:50.082641 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7491-140675008579312"] Jul 24 15:41:50.083115 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:50.083099 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6acb9fe8-d357-4e1f-9b42-acd8907d159c" containerName="test-curl-7491-140675008579312" Jul 24 15:41:50.083170 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:50.083118 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="6acb9fe8-d357-4e1f-9b42-acd8907d159c" containerName="test-curl-7491-140675008579312" Jul 24 15:41:50.083218 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:50.083206 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="6acb9fe8-d357-4e1f-9b42-acd8907d159c" containerName="test-curl-7491-140675008579312" Jul 24 15:41:50.086303 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:50.086284 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675008579312" Jul 24 15:41:50.093267 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:50.093238 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140675008579312"] Jul 24 15:41:50.103846 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:50.103810 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dp886\" (UniqueName: \"kubernetes.io/projected/6acb9fe8-d357-4e1f-9b42-acd8907d159c-kube-api-access-dp886\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:41:50.204838 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:50.204797 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skd8v\" (UniqueName: \"kubernetes.io/projected/4b93a410-fdfd-4b10-9026-a7b14bb759f2-kube-api-access-skd8v\") pod \"test-curl-7491-140675008579312\" (UID: \"4b93a410-fdfd-4b10-9026-a7b14bb759f2\") " pod="opendatahub/test-curl-7491-140675008579312" Jul 24 15:41:50.277998 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:50.277960 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6acb9fe8-d357-4e1f-9b42-acd8907d159c" path="/var/lib/kubelet/pods/6acb9fe8-d357-4e1f-9b42-acd8907d159c/volumes" Jul 24 15:41:50.305670 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:50.305626 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skd8v\" (UniqueName: \"kubernetes.io/projected/4b93a410-fdfd-4b10-9026-a7b14bb759f2-kube-api-access-skd8v\") pod \"test-curl-7491-140675008579312\" (UID: \"4b93a410-fdfd-4b10-9026-a7b14bb759f2\") " pod="opendatahub/test-curl-7491-140675008579312" Jul 24 15:41:50.313137 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:50.313110 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skd8v\" (UniqueName: \"kubernetes.io/projected/4b93a410-fdfd-4b10-9026-a7b14bb759f2-kube-api-access-skd8v\") pod \"test-curl-7491-140675008579312\" (UID: \"4b93a410-fdfd-4b10-9026-a7b14bb759f2\") " pod="opendatahub/test-curl-7491-140675008579312" Jul 24 15:41:50.398418 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:50.398328 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675008579312" Jul 24 15:41:50.545088 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:50.545057 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140675008579312"] Jul 24 15:41:50.547086 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:41:50.547059 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b93a410_fdfd_4b10_9026_a7b14bb759f2.slice/crio-e7949cbc73e50098be8c85f52751543f6b05950499f7644f4b5f5a644e9b277d WatchSource:0}: Error finding container e7949cbc73e50098be8c85f52751543f6b05950499f7644f4b5f5a644e9b277d: Status 404 returned error can't find the container with id e7949cbc73e50098be8c85f52751543f6b05950499f7644f4b5f5a644e9b277d Jul 24 15:41:50.770249 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:50.770214 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675008579312" Jul 24 15:41:50.770249 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:50.770241 2531 scope.go:117] "RemoveContainer" containerID="38af35fff13b89db17aae506b04d7b69a0dfd74a01d84dcfd40856d8ff78e3df" Jul 24 15:41:50.771646 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:50.771619 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140675008579312" event={"ID":"4b93a410-fdfd-4b10-9026-a7b14bb759f2","Type":"ContainerStarted","Data":"e7949cbc73e50098be8c85f52751543f6b05950499f7644f4b5f5a644e9b277d"} Jul 24 15:41:51.780288 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:51.780247 2531 generic.go:356] "Generic (PLEG): container finished" podID="4b93a410-fdfd-4b10-9026-a7b14bb759f2" containerID="9342de24e8c994bf6926e1737221c39861500012a87a99b41bf8746cd79e29ef" exitCode=0 Jul 24 15:41:51.780750 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:51.780335 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140675008579312" event={"ID":"4b93a410-fdfd-4b10-9026-a7b14bb759f2","Type":"ContainerDied","Data":"9342de24e8c994bf6926e1737221c39861500012a87a99b41bf8746cd79e29ef"} Jul 24 15:41:52.935851 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:52.935820 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675008579312" Jul 24 15:41:52.956614 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:52.956588 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7491-140675008579312_4b93a410-fdfd-4b10-9026-a7b14bb759f2/test-curl-7491-140675008579312/0.log" Jul 24 15:41:52.983526 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:52.983460 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7491-140675008579312"] Jul 24 15:41:52.986611 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:52.986579 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7491-140675008579312"] Jul 24 15:41:53.032697 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:53.032665 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skd8v\" (UniqueName: \"kubernetes.io/projected/4b93a410-fdfd-4b10-9026-a7b14bb759f2-kube-api-access-skd8v\") pod \"4b93a410-fdfd-4b10-9026-a7b14bb759f2\" (UID: \"4b93a410-fdfd-4b10-9026-a7b14bb759f2\") " Jul 24 15:41:53.035032 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:53.035002 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b93a410-fdfd-4b10-9026-a7b14bb759f2-kube-api-access-skd8v" (OuterVolumeSpecName: "kube-api-access-skd8v") pod "4b93a410-fdfd-4b10-9026-a7b14bb759f2" (UID: "4b93a410-fdfd-4b10-9026-a7b14bb759f2"). InnerVolumeSpecName "kube-api-access-skd8v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:41:53.105239 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:53.105200 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7491-140675006760400"] Jul 24 15:41:53.105634 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:53.105622 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b93a410-fdfd-4b10-9026-a7b14bb759f2" containerName="test-curl-7491-140675008579312" Jul 24 15:41:53.105680 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:53.105636 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b93a410-fdfd-4b10-9026-a7b14bb759f2" containerName="test-curl-7491-140675008579312" Jul 24 15:41:53.105732 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:53.105723 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b93a410-fdfd-4b10-9026-a7b14bb759f2" containerName="test-curl-7491-140675008579312" Jul 24 15:41:53.109092 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:53.109074 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675006760400" Jul 24 15:41:53.113320 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:53.113292 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140675006760400"] Jul 24 15:41:53.133489 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:53.133457 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skd8v\" (UniqueName: \"kubernetes.io/projected/4b93a410-fdfd-4b10-9026-a7b14bb759f2-kube-api-access-skd8v\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:41:53.234506 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:53.234473 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzp89\" (UniqueName: \"kubernetes.io/projected/cd641c65-7592-465b-ac47-c2b99ba272e4-kube-api-access-dzp89\") pod \"test-curl-7491-140675006760400\" (UID: \"cd641c65-7592-465b-ac47-c2b99ba272e4\") " pod="opendatahub/test-curl-7491-140675006760400" Jul 24 15:41:53.335597 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:53.335486 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzp89\" (UniqueName: \"kubernetes.io/projected/cd641c65-7592-465b-ac47-c2b99ba272e4-kube-api-access-dzp89\") pod \"test-curl-7491-140675006760400\" (UID: \"cd641c65-7592-465b-ac47-c2b99ba272e4\") " pod="opendatahub/test-curl-7491-140675006760400" Jul 24 15:41:53.344052 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:53.344018 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzp89\" (UniqueName: \"kubernetes.io/projected/cd641c65-7592-465b-ac47-c2b99ba272e4-kube-api-access-dzp89\") pod \"test-curl-7491-140675006760400\" (UID: \"cd641c65-7592-465b-ac47-c2b99ba272e4\") " pod="opendatahub/test-curl-7491-140675006760400" Jul 24 15:41:53.421128 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:53.421090 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675006760400" Jul 24 15:41:53.568150 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:53.568119 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140675006760400"] Jul 24 15:41:53.569740 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:41:53.569712 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd641c65_7592_465b_ac47_c2b99ba272e4.slice/crio-df996fcb2a98bbe83ba84677e50db11cf6b4ef688f418cc330f6cc965d6e232d WatchSource:0}: Error finding container df996fcb2a98bbe83ba84677e50db11cf6b4ef688f418cc330f6cc965d6e232d: Status 404 returned error can't find the container with id df996fcb2a98bbe83ba84677e50db11cf6b4ef688f418cc330f6cc965d6e232d Jul 24 15:41:53.794746 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:53.794715 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675008579312" Jul 24 15:41:53.794746 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:53.794743 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e7949cbc73e50098be8c85f52751543f6b05950499f7644f4b5f5a644e9b277d" Jul 24 15:41:53.796047 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:53.796017 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140675006760400" event={"ID":"cd641c65-7592-465b-ac47-c2b99ba272e4","Type":"ContainerStarted","Data":"df996fcb2a98bbe83ba84677e50db11cf6b4ef688f418cc330f6cc965d6e232d"} Jul 24 15:41:53.805040 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:41:53.805014 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7491-140675008579312\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="4b93a410-fdfd-4b10-9026-a7b14bb759f2" pod="opendatahub/test-curl-7491-140675008579312" Jul 24 15:41:54.278344 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:54.278313 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b93a410-fdfd-4b10-9026-a7b14bb759f2" path="/var/lib/kubelet/pods/4b93a410-fdfd-4b10-9026-a7b14bb759f2/volumes" Jul 24 15:41:54.803944 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:54.803905 2531 generic.go:356] "Generic (PLEG): container finished" podID="cd641c65-7592-465b-ac47-c2b99ba272e4" containerID="a42c08d0cc818d63356a63309d38fce25fb20d7ffada41676ed5f775d18a7cf4" exitCode=0 Jul 24 15:41:54.804133 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:54.803999 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140675006760400" event={"ID":"cd641c65-7592-465b-ac47-c2b99ba272e4","Type":"ContainerDied","Data":"a42c08d0cc818d63356a63309d38fce25fb20d7ffada41676ed5f775d18a7cf4"} Jul 24 15:41:55.965722 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:55.965696 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675006760400" Jul 24 15:41:55.984666 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:55.984635 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7491-140675006760400_cd641c65-7592-465b-ac47-c2b99ba272e4/test-curl-7491-140675006760400/0.log" Jul 24 15:41:56.009116 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:56.009079 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7491-140675006760400"] Jul 24 15:41:56.011565 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:56.011539 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7491-140675006760400"] Jul 24 15:41:56.063524 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:56.063421 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzp89\" (UniqueName: \"kubernetes.io/projected/cd641c65-7592-465b-ac47-c2b99ba272e4-kube-api-access-dzp89\") pod \"cd641c65-7592-465b-ac47-c2b99ba272e4\" (UID: \"cd641c65-7592-465b-ac47-c2b99ba272e4\") " Jul 24 15:41:56.065762 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:56.065723 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd641c65-7592-465b-ac47-c2b99ba272e4-kube-api-access-dzp89" (OuterVolumeSpecName: "kube-api-access-dzp89") pod "cd641c65-7592-465b-ac47-c2b99ba272e4" (UID: "cd641c65-7592-465b-ac47-c2b99ba272e4"). InnerVolumeSpecName "kube-api-access-dzp89". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:41:56.133054 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:56.133019 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7491-140675006760400"] Jul 24 15:41:56.133475 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:56.133462 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd641c65-7592-465b-ac47-c2b99ba272e4" containerName="test-curl-7491-140675006760400" Jul 24 15:41:56.133520 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:56.133477 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd641c65-7592-465b-ac47-c2b99ba272e4" containerName="test-curl-7491-140675006760400" Jul 24 15:41:56.133565 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:56.133556 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd641c65-7592-465b-ac47-c2b99ba272e4" containerName="test-curl-7491-140675006760400" Jul 24 15:41:56.140960 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:56.137426 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675006760400" Jul 24 15:41:56.144852 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:56.144822 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140675006760400"] Jul 24 15:41:56.164434 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:56.164385 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzp89\" (UniqueName: \"kubernetes.io/projected/cd641c65-7592-465b-ac47-c2b99ba272e4-kube-api-access-dzp89\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:41:56.265595 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:56.265546 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vn6p\" (UniqueName: \"kubernetes.io/projected/984dc934-6c54-4754-95a2-8d7e85e63694-kube-api-access-7vn6p\") pod \"test-curl-7491-140675006760400\" (UID: \"984dc934-6c54-4754-95a2-8d7e85e63694\") " pod="opendatahub/test-curl-7491-140675006760400" Jul 24 15:41:56.278496 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:56.278457 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd641c65-7592-465b-ac47-c2b99ba272e4" path="/var/lib/kubelet/pods/cd641c65-7592-465b-ac47-c2b99ba272e4/volumes" Jul 24 15:41:56.367196 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:56.367102 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vn6p\" (UniqueName: \"kubernetes.io/projected/984dc934-6c54-4754-95a2-8d7e85e63694-kube-api-access-7vn6p\") pod \"test-curl-7491-140675006760400\" (UID: \"984dc934-6c54-4754-95a2-8d7e85e63694\") " pod="opendatahub/test-curl-7491-140675006760400" Jul 24 15:41:56.375030 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:56.374995 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vn6p\" (UniqueName: \"kubernetes.io/projected/984dc934-6c54-4754-95a2-8d7e85e63694-kube-api-access-7vn6p\") pod \"test-curl-7491-140675006760400\" (UID: \"984dc934-6c54-4754-95a2-8d7e85e63694\") " pod="opendatahub/test-curl-7491-140675006760400" Jul 24 15:41:56.456115 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:56.456070 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675006760400" Jul 24 15:41:56.605589 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:56.605559 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7491-140675006760400"] Jul 24 15:41:56.607194 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:41:56.607165 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod984dc934_6c54_4754_95a2_8d7e85e63694.slice/crio-bcaf20c2e12dbe45a44e545b7f64b9c78abb3e34321c53ff48cc0c3ebbb6086f WatchSource:0}: Error finding container bcaf20c2e12dbe45a44e545b7f64b9c78abb3e34321c53ff48cc0c3ebbb6086f: Status 404 returned error can't find the container with id bcaf20c2e12dbe45a44e545b7f64b9c78abb3e34321c53ff48cc0c3ebbb6086f Jul 24 15:41:56.817531 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:56.817499 2531 scope.go:117] "RemoveContainer" containerID="a42c08d0cc818d63356a63309d38fce25fb20d7ffada41676ed5f775d18a7cf4" Jul 24 15:41:56.817720 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:56.817507 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675006760400" Jul 24 15:41:56.818925 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:56.818897 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140675006760400" event={"ID":"984dc934-6c54-4754-95a2-8d7e85e63694","Type":"ContainerStarted","Data":"bcaf20c2e12dbe45a44e545b7f64b9c78abb3e34321c53ff48cc0c3ebbb6086f"} Jul 24 15:41:57.827579 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:57.827547 2531 generic.go:356] "Generic (PLEG): container finished" podID="984dc934-6c54-4754-95a2-8d7e85e63694" containerID="ff2c14755510c244bc6422c34db1c182cf4ca42b4bd0d25807cf476028d52641" exitCode=0 Jul 24 15:41:57.828000 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:57.827639 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7491-140675006760400" event={"ID":"984dc934-6c54-4754-95a2-8d7e85e63694","Type":"ContainerDied","Data":"ff2c14755510c244bc6422c34db1c182cf4ca42b4bd0d25807cf476028d52641"} Jul 24 15:41:58.976926 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:58.976899 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675006760400" Jul 24 15:41:58.996668 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:58.996639 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7491-140675006760400_984dc934-6c54-4754-95a2-8d7e85e63694/test-curl-7491-140675006760400/0.log" Jul 24 15:41:59.023094 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:59.023055 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7491-140675006760400"] Jul 24 15:41:59.027378 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:59.027347 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7491-140675006760400"] Jul 24 15:41:59.095312 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:59.095219 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vn6p\" (UniqueName: \"kubernetes.io/projected/984dc934-6c54-4754-95a2-8d7e85e63694-kube-api-access-7vn6p\") pod \"984dc934-6c54-4754-95a2-8d7e85e63694\" (UID: \"984dc934-6c54-4754-95a2-8d7e85e63694\") " Jul 24 15:41:59.097439 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:59.097405 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/984dc934-6c54-4754-95a2-8d7e85e63694-kube-api-access-7vn6p" (OuterVolumeSpecName: "kube-api-access-7vn6p") pod "984dc934-6c54-4754-95a2-8d7e85e63694" (UID: "984dc934-6c54-4754-95a2-8d7e85e63694"). InnerVolumeSpecName "kube-api-access-7vn6p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:41:59.196833 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:59.196793 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vn6p\" (UniqueName: \"kubernetes.io/projected/984dc934-6c54-4754-95a2-8d7e85e63694-kube-api-access-7vn6p\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:41:59.541097 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:59.541061 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:41:59.843329 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:59.843237 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bcaf20c2e12dbe45a44e545b7f64b9c78abb3e34321c53ff48cc0c3ebbb6086f" Jul 24 15:41:59.843329 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:59.843244 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7491-140675006760400" Jul 24 15:41:59.853796 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:41:59.853745 2531 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7491-140675006760400\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="984dc934-6c54-4754-95a2-8d7e85e63694" pod="opendatahub/test-curl-7491-140675006760400" Jul 24 15:41:59.888458 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:41:59.888420 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:00.282902 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:00.282826 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="984dc934-6c54-4754-95a2-8d7e85e63694" path="/var/lib/kubelet/pods/984dc934-6c54-4754-95a2-8d7e85e63694/volumes" Jul 24 15:42:00.337909 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:00.337822 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:02.773763 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:02.773726 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:02.884074 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:02.884042 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:03.606413 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:03.606375 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:03.876473 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:03.876386 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:04.216524 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:04.216493 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:04.311996 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:04.311962 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:07.286895 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:07.286839 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:11.559479 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:11.559436 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:42:18.248405 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.248370 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:18.565680 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.565600 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c"] Jul 24 15:42:18.566120 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.566104 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="984dc934-6c54-4754-95a2-8d7e85e63694" containerName="test-curl-7491-140675006760400" Jul 24 15:42:18.566171 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.566122 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="984dc934-6c54-4754-95a2-8d7e85e63694" containerName="test-curl-7491-140675006760400" Jul 24 15:42:18.566208 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.566196 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="984dc934-6c54-4754-95a2-8d7e85e63694" containerName="test-curl-7491-140675006760400" Jul 24 15:42:18.569512 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.569495 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.571659 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.571632 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-053e45-openshift-default-dockercfg-d4vfb\"" Jul 24 15:42:18.580060 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.580024 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-workload-socket\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.580311 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.580286 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-credential-socket\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.580428 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.580353 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-envoy\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.580428 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.580399 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istiod-ca-cert\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.580538 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.580443 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h68fk\" (UniqueName: \"kubernetes.io/projected/edc17d5c-7b1e-4196-9594-0d984bb8a10f-kube-api-access-h68fk\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.580538 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.580507 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-workload-certs\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.580623 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.580556 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-podinfo\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.580623 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.580607 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-token\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.580709 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.580648 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c"] Jul 24 15:42:18.580748 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.580724 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-data\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.597406 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.597364 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:18.681280 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.681243 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-data\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.681464 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.681299 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-workload-socket\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.681464 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.681323 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-credential-socket\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.681464 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.681340 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-envoy\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.681464 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.681354 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istiod-ca-cert\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.681464 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.681382 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h68fk\" (UniqueName: \"kubernetes.io/projected/edc17d5c-7b1e-4196-9594-0d984bb8a10f-kube-api-access-h68fk\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.681464 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.681409 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-workload-certs\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.681831 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.681544 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-podinfo\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.681831 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.681607 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-token\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.681831 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.681725 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-data\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.681831 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.681782 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-credential-socket\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.682056 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.681895 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-workload-socket\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.682056 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.681918 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-workload-certs\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.682309 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.682290 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istiod-ca-cert\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.683987 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.683965 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-envoy\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.684191 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.684174 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-podinfo\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.689911 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.689860 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-token\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.690107 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.690088 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h68fk\" (UniqueName: \"kubernetes.io/projected/edc17d5c-7b1e-4196-9594-0d984bb8a10f-kube-api-access-h68fk\") pod \"e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:18.881843 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:18.881744 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:19.052536 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:19.050895 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c"] Jul 24 15:42:19.055204 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:19.055171 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 24 15:42:19.055316 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:19.055238 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 24 15:42:19.055316 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:19.055266 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 24 15:42:19.085575 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:19.085517 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:19.975693 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:19.975659 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" event={"ID":"edc17d5c-7b1e-4196-9594-0d984bb8a10f","Type":"ContainerStarted","Data":"b9cea02b3ff3d0719088b835e7d4d68d2ff9f86c748cd192549cfff0a1991e3f"} Jul 24 15:42:19.976218 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:19.975718 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" event={"ID":"edc17d5c-7b1e-4196-9594-0d984bb8a10f","Type":"ContainerStarted","Data":"ba34b5c68dca49e57d60dc13d309cd4f27ff089183868fcaef361afcb43ea0fe"} Jul 24 15:42:19.995203 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:19.995152 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" podStartSLOduration=1.995137887 podStartE2EDuration="1.995137887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:42:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:42:19.994091154 +0000 UTC m=+2772.224567838" watchObservedRunningTime="2026-07-24 15:42:19.995137887 +0000 UTC m=+2772.225614571" Jul 24 15:42:20.882742 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:20.882704 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:20.888404 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:20.888374 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:20.982482 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:20.982450 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:20.983561 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:20.983541 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:21.938456 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:21.938419 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:22.038261 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:22.038227 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:23.760320 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:23.760284 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968"] Jul 24 15:42:23.764420 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:23.764394 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" Jul 24 15:42:23.766658 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:23.766629 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-053e45-serving-cert\"" Jul 24 15:42:23.766803 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:23.766690 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-n8r5x\"" Jul 24 15:42:23.775638 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:23.775608 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968"] Jul 24 15:42:23.794350 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:23.794310 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:23.824054 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:23.824016 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q89d\" (UniqueName: \"kubernetes.io/projected/c7d84236-d9b8-4793-a83a-810a04c016db-kube-api-access-8q89d\") pod \"maas-api-e2e-ipp-b-053e45-6846f84978-4m968\" (UID: \"c7d84236-d9b8-4793-a83a-810a04c016db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" Jul 24 15:42:23.824246 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:23.824075 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7d84236-d9b8-4793-a83a-810a04c016db-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-053e45-6846f84978-4m968\" (UID: \"c7d84236-d9b8-4793-a83a-810a04c016db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" Jul 24 15:42:23.824246 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:23.824193 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7d84236-d9b8-4793-a83a-810a04c016db-maas-api-tls\") pod \"maas-api-e2e-ipp-b-053e45-6846f84978-4m968\" (UID: \"c7d84236-d9b8-4793-a83a-810a04c016db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" Jul 24 15:42:23.925606 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:23.925564 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q89d\" (UniqueName: \"kubernetes.io/projected/c7d84236-d9b8-4793-a83a-810a04c016db-kube-api-access-8q89d\") pod \"maas-api-e2e-ipp-b-053e45-6846f84978-4m968\" (UID: \"c7d84236-d9b8-4793-a83a-810a04c016db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" Jul 24 15:42:23.925821 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:23.925619 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7d84236-d9b8-4793-a83a-810a04c016db-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-053e45-6846f84978-4m968\" (UID: \"c7d84236-d9b8-4793-a83a-810a04c016db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" Jul 24 15:42:23.925821 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:23.925656 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7d84236-d9b8-4793-a83a-810a04c016db-maas-api-tls\") pod \"maas-api-e2e-ipp-b-053e45-6846f84978-4m968\" (UID: \"c7d84236-d9b8-4793-a83a-810a04c016db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" Jul 24 15:42:23.925821 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:42:23.925786 2531 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-serving-cert: secret "maas-api-e2e-ipp-b-053e45-serving-cert" not found Jul 24 15:42:23.926068 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:42:23.925838 2531 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7d84236-d9b8-4793-a83a-810a04c016db-maas-api-tls podName:c7d84236-d9b8-4793-a83a-810a04c016db nodeName:}" failed. No retries permitted until 2026-07-24 15:42:24.425822845 +0000 UTC m=+2776.656299506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c7d84236-d9b8-4793-a83a-810a04c016db-maas-api-tls") pod "maas-api-e2e-ipp-b-053e45-6846f84978-4m968" (UID: "c7d84236-d9b8-4793-a83a-810a04c016db") : secret "maas-api-e2e-ipp-b-053e45-serving-cert" not found Jul 24 15:42:23.926438 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:23.926412 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7d84236-d9b8-4793-a83a-810a04c016db-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-053e45-6846f84978-4m968\" (UID: \"c7d84236-d9b8-4793-a83a-810a04c016db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" Jul 24 15:42:23.934440 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:23.934402 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q89d\" (UniqueName: \"kubernetes.io/projected/c7d84236-d9b8-4793-a83a-810a04c016db-kube-api-access-8q89d\") pod \"maas-api-e2e-ipp-b-053e45-6846f84978-4m968\" (UID: \"c7d84236-d9b8-4793-a83a-810a04c016db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" Jul 24 15:42:24.135144 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:24.135058 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:24.283179 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:24.283140 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:24.384885 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:24.384835 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:24.431442 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:24.431399 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7d84236-d9b8-4793-a83a-810a04c016db-maas-api-tls\") pod \"maas-api-e2e-ipp-b-053e45-6846f84978-4m968\" (UID: \"c7d84236-d9b8-4793-a83a-810a04c016db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" Jul 24 15:42:24.434298 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:24.434270 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7d84236-d9b8-4793-a83a-810a04c016db-maas-api-tls\") pod \"maas-api-e2e-ipp-b-053e45-6846f84978-4m968\" (UID: \"c7d84236-d9b8-4793-a83a-810a04c016db\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" Jul 24 15:42:24.680957 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:24.680917 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" Jul 24 15:42:25.043097 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:25.043062 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968"] Jul 24 15:42:25.044362 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:42:25.044333 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7d84236_d9b8_4793_a83a_810a04c016db.slice/crio-7d944eb9ecba526b9b7cc3dd4a4b5dde88ad405728bd3cf44885930496ea7c7b WatchSource:0}: Error finding container 7d944eb9ecba526b9b7cc3dd4a4b5dde88ad405728bd3cf44885930496ea7c7b: Status 404 returned error can't find the container with id 7d944eb9ecba526b9b7cc3dd4a4b5dde88ad405728bd3cf44885930496ea7c7b Jul 24 15:42:26.023259 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:26.023221 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" event={"ID":"c7d84236-d9b8-4793-a83a-810a04c016db","Type":"ContainerStarted","Data":"7c7aa4298f7e8e4971fc65549630beef97e1342c13615dad95db5ff50773b3e4"} Jul 24 15:42:26.023479 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:26.023277 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" event={"ID":"c7d84236-d9b8-4793-a83a-810a04c016db","Type":"ContainerStarted","Data":"7d944eb9ecba526b9b7cc3dd4a4b5dde88ad405728bd3cf44885930496ea7c7b"} Jul 24 15:42:26.023479 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:26.023438 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" Jul 24 15:42:26.038712 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:26.038655 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" podStartSLOduration=2.722093817 podStartE2EDuration="3.038636149s" totalImagesPullingTime="316.542332ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:42:26.037518717 +0000 UTC m=+2778.267995426" watchObservedRunningTime="2026-07-24 15:42:26.038636149 +0000 UTC m=+2778.269112836" Jul 24 15:42:26.493075 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:26.493030 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:30.964327 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:30.964241 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:42:32.038419 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:32.038390 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" Jul 24 15:42:40.134247 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:40.134212 2531 generic.go:356] "Generic (PLEG): container finished" podID="edc17d5c-7b1e-4196-9594-0d984bb8a10f" containerID="b9cea02b3ff3d0719088b835e7d4d68d2ff9f86c748cd192549cfff0a1991e3f" exitCode=0 Jul 24 15:42:40.134798 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:40.134288 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" event={"ID":"edc17d5c-7b1e-4196-9594-0d984bb8a10f","Type":"ContainerDied","Data":"b9cea02b3ff3d0719088b835e7d4d68d2ff9f86c748cd192549cfff0a1991e3f"} Jul 24 15:42:40.134932 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:40.134913 2531 scope.go:117] "RemoveContainer" containerID="b9cea02b3ff3d0719088b835e7d4d68d2ff9f86c748cd192549cfff0a1991e3f" Jul 24 15:42:40.136091 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:40.136062 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 24 15:42:40.136158 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:40.136137 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 24 15:42:40.136197 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:40.136169 2531 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"} Jul 24 15:42:41.142175 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:41.142136 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" event={"ID":"edc17d5c-7b1e-4196-9594-0d984bb8a10f","Type":"ContainerStarted","Data":"7511d6706c7ab5c5e5f32ff9897bb92fc70f968b2917746536f8dff2f476ee9c"} Jul 24 15:42:41.882260 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:41.882208 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:41.882604 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:41.882569 2531 patch_prober.go:28] interesting pod/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 24 15:42:41.882711 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:41.882654 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" podUID="edc17d5c-7b1e-4196-9594-0d984bb8a10f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 24 15:42:42.882623 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:42.882574 2531 patch_prober.go:28] interesting pod/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 24 15:42:42.883166 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:42.882648 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" podUID="edc17d5c-7b1e-4196-9594-0d984bb8a10f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 24 15:42:43.752929 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:43.752900 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-hfcxx_58138eae-76c2-489f-abdb-f858b5ef0108/payload-processing/0.log" Jul 24 15:42:43.883126 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:43.883086 2531 patch_prober.go:28] interesting pod/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 24 15:42:43.883633 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:43.883171 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" podUID="edc17d5c-7b1e-4196-9594-0d984bb8a10f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 24 15:42:44.882969 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:44.882925 2531 patch_prober.go:28] interesting pod/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 24 15:42:44.883197 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:44.883002 2531 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" podUID="edc17d5c-7b1e-4196-9594-0d984bb8a10f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 24 15:42:45.886470 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:45.886430 2531 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:45.886964 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:45.886896 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:45.887798 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:45.887777 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:42:50.578897 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:50.578834 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:42:54.652199 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:42:54.652151 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:43:24.253005 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:24.252960 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:43:36.005347 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:36.005312 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-hfcxx_58138eae-76c2-489f-abdb-f858b5ef0108/payload-processing/0.log" Jul 24 15:43:36.462139 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:36.462104 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:36.785542 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:36.785440 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:37.146924 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:37.146818 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:40.242051 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:40.242009 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:40.341992 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:40.341957 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:42.637850 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:42.637811 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr"] Jul 24 15:43:42.642049 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:42.642019 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" Jul 24 15:43:42.644097 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:42.644071 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-923368-serving-cert\"" Jul 24 15:43:42.652501 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:42.652459 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr"] Jul 24 15:43:42.669086 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:42.669041 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:42.731257 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:42.731219 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8121c744-662a-4669-a7f0-9957f241f60d-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr\" (UID: \"8121c744-662a-4669-a7f0-9957f241f60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" Jul 24 15:43:42.731443 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:42.731300 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9z2v\" (UniqueName: \"kubernetes.io/projected/8121c744-662a-4669-a7f0-9957f241f60d-kube-api-access-t9z2v\") pod \"maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr\" (UID: \"8121c744-662a-4669-a7f0-9957f241f60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" Jul 24 15:43:42.731443 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:42.731339 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8121c744-662a-4669-a7f0-9957f241f60d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr\" (UID: \"8121c744-662a-4669-a7f0-9957f241f60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" Jul 24 15:43:42.781157 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:42.781124 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:42.832752 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:42.832715 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8121c744-662a-4669-a7f0-9957f241f60d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr\" (UID: \"8121c744-662a-4669-a7f0-9957f241f60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" Jul 24 15:43:42.832964 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:42.832814 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8121c744-662a-4669-a7f0-9957f241f60d-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr\" (UID: \"8121c744-662a-4669-a7f0-9957f241f60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" Jul 24 15:43:42.832964 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:42.832888 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9z2v\" (UniqueName: \"kubernetes.io/projected/8121c744-662a-4669-a7f0-9957f241f60d-kube-api-access-t9z2v\") pod \"maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr\" (UID: \"8121c744-662a-4669-a7f0-9957f241f60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" Jul 24 15:43:42.833478 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:42.833451 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8121c744-662a-4669-a7f0-9957f241f60d-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr\" (UID: \"8121c744-662a-4669-a7f0-9957f241f60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" Jul 24 15:43:42.835350 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:42.835323 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8121c744-662a-4669-a7f0-9957f241f60d-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr\" (UID: \"8121c744-662a-4669-a7f0-9957f241f60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" Jul 24 15:43:42.840347 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:42.840321 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9z2v\" (UniqueName: \"kubernetes.io/projected/8121c744-662a-4669-a7f0-9957f241f60d-kube-api-access-t9z2v\") pod \"maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr\" (UID: \"8121c744-662a-4669-a7f0-9957f241f60d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" Jul 24 15:43:42.954595 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:42.954562 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" Jul 24 15:43:43.101525 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:43.101497 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr"] Jul 24 15:43:43.103813 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:43:43.103633 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8121c744_662a_4669_a7f0_9957f241f60d.slice/crio-c9a614f4606fac146968c2cce2c081ea3d3e41760ac315fb94c319f0af0c808c WatchSource:0}: Error finding container c9a614f4606fac146968c2cce2c081ea3d3e41760ac315fb94c319f0af0c808c: Status 404 returned error can't find the container with id c9a614f4606fac146968c2cce2c081ea3d3e41760ac315fb94c319f0af0c808c Jul 24 15:43:43.110901 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:43.110845 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:43.209345 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:43.209254 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:43.543012 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:43.542923 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" event={"ID":"8121c744-662a-4669-a7f0-9957f241f60d","Type":"ContainerStarted","Data":"c9a614f4606fac146968c2cce2c081ea3d3e41760ac315fb94c319f0af0c808c"} Jul 24 15:43:44.550623 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:44.550587 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" event={"ID":"8121c744-662a-4669-a7f0-9957f241f60d","Type":"ContainerStarted","Data":"3de89cc96d62d8cd20e4600ed10eae8e3d9f5d0069995438719efe52c28a76b9"} Jul 24 15:43:44.551064 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:44.550709 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" Jul 24 15:43:44.568945 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:44.568885 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" podStartSLOduration=2.123669352 podStartE2EDuration="2.568855001s" totalImagesPullingTime="445.185649ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:43:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:43:44.56578339 +0000 UTC m=+2856.796260074" watchObservedRunningTime="2026-07-24 15:43:44.568855001 +0000 UTC m=+2856.799331686" Jul 24 15:43:44.792857 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:44.792805 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:43:50.564181 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:50.564150 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" Jul 24 15:43:51.964406 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:51.964363 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:43:55.753687 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:55.753650 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-923368-gxv4k"] Jul 24 15:43:55.757587 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:55.757561 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-923368-gxv4k" Jul 24 15:43:55.759979 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:55.759958 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q8grb\"" Jul 24 15:43:55.765668 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:55.765639 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-923368-gxv4k"] Jul 24 15:43:55.867507 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:55.867470 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42fd2aef-5f3d-45e7-9b75-1aaf49e583a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-923368-gxv4k\" (UID: \"42fd2aef-5f3d-45e7-9b75-1aaf49e583a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-923368-gxv4k" Jul 24 15:43:55.968110 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:55.968070 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42fd2aef-5f3d-45e7-9b75-1aaf49e583a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-923368-gxv4k\" (UID: \"42fd2aef-5f3d-45e7-9b75-1aaf49e583a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-923368-gxv4k" Jul 24 15:43:55.968740 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:55.968718 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42fd2aef-5f3d-45e7-9b75-1aaf49e583a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-923368-gxv4k\" (UID: \"42fd2aef-5f3d-45e7-9b75-1aaf49e583a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-923368-gxv4k" Jul 24 15:43:56.071339 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:56.071247 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-923368-gxv4k" Jul 24 15:43:56.222600 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:56.222575 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-923368-gxv4k"] Jul 24 15:43:56.224770 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:43:56.224741 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42fd2aef_5f3d_45e7_9b75_1aaf49e583a0.slice/crio-7eaa9d320224a12617319909f6d86736835adbc62d9c81b8c8b42b2efba2a2d2 WatchSource:0}: Error finding container 7eaa9d320224a12617319909f6d86736835adbc62d9c81b8c8b42b2efba2a2d2: Status 404 returned error can't find the container with id 7eaa9d320224a12617319909f6d86736835adbc62d9c81b8c8b42b2efba2a2d2 Jul 24 15:43:56.632454 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:56.632418 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-923368-gxv4k" event={"ID":"42fd2aef-5f3d-45e7-9b75-1aaf49e583a0","Type":"ContainerStarted","Data":"04c1c6bd279d4643fac7feed589e2422baaf3aab4bb2870e8a895478480051b3"} Jul 24 15:43:56.632631 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:56.632473 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-923368-gxv4k" event={"ID":"42fd2aef-5f3d-45e7-9b75-1aaf49e583a0","Type":"ContainerStarted","Data":"7eaa9d320224a12617319909f6d86736835adbc62d9c81b8c8b42b2efba2a2d2"} Jul 24 15:43:56.649795 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:56.649740 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-923368-gxv4k" podStartSLOduration=1.649721217 podStartE2EDuration="1.649721217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:43:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:43:56.648630324 +0000 UTC m=+2868.879107204" watchObservedRunningTime="2026-07-24 15:43:56.649721217 +0000 UTC m=+2868.880197900" Jul 24 15:43:57.640542 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:57.640504 2531 generic.go:356] "Generic (PLEG): container finished" podID="42fd2aef-5f3d-45e7-9b75-1aaf49e583a0" containerID="04c1c6bd279d4643fac7feed589e2422baaf3aab4bb2870e8a895478480051b3" exitCode=0 Jul 24 15:43:57.640954 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:57.640595 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-923368-gxv4k" event={"ID":"42fd2aef-5f3d-45e7-9b75-1aaf49e583a0","Type":"ContainerDied","Data":"04c1c6bd279d4643fac7feed589e2422baaf3aab4bb2870e8a895478480051b3"} Jul 24 15:43:58.796600 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:58.796574 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-923368-gxv4k" Jul 24 15:43:58.897430 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:58.897395 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42fd2aef-5f3d-45e7-9b75-1aaf49e583a0-maas-api-service-ca\") pod \"42fd2aef-5f3d-45e7-9b75-1aaf49e583a0\" (UID: \"42fd2aef-5f3d-45e7-9b75-1aaf49e583a0\") " Jul 24 15:43:58.897793 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:58.897744 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42fd2aef-5f3d-45e7-9b75-1aaf49e583a0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "42fd2aef-5f3d-45e7-9b75-1aaf49e583a0" (UID: "42fd2aef-5f3d-45e7-9b75-1aaf49e583a0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:43:58.999143 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:58.999105 2531 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42fd2aef-5f3d-45e7-9b75-1aaf49e583a0-maas-api-service-ca\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:43:59.655221 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:59.655183 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-923368-gxv4k" event={"ID":"42fd2aef-5f3d-45e7-9b75-1aaf49e583a0","Type":"ContainerDied","Data":"7eaa9d320224a12617319909f6d86736835adbc62d9c81b8c8b42b2efba2a2d2"} Jul 24 15:43:59.655221 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:59.655209 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-923368-gxv4k" Jul 24 15:43:59.655439 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:43:59.655270 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7eaa9d320224a12617319909f6d86736835adbc62d9c81b8c8b42b2efba2a2d2" Jul 24 15:44:09.522533 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:09.522480 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr"] Jul 24 15:44:09.523165 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:09.522750 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" podUID="8121c744-662a-4669-a7f0-9957f241f60d" containerName="maas-api" containerID="cri-o://3de89cc96d62d8cd20e4600ed10eae8e3d9f5d0069995438719efe52c28a76b9" gracePeriod=30 Jul 24 15:44:09.552045 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:09.551997 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:09.598026 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:44:09.597982 2531 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8121c744_662a_4669_a7f0_9957f241f60d.slice/crio-conmon-3de89cc96d62d8cd20e4600ed10eae8e3d9f5d0069995438719efe52c28a76b9.scope\": RecentStats: unable to find data in memory cache]" Jul 24 15:44:09.731498 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:09.731433 2531 generic.go:356] "Generic (PLEG): container finished" podID="8121c744-662a-4669-a7f0-9957f241f60d" containerID="3de89cc96d62d8cd20e4600ed10eae8e3d9f5d0069995438719efe52c28a76b9" exitCode=0 Jul 24 15:44:09.731755 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:09.731704 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" event={"ID":"8121c744-662a-4669-a7f0-9957f241f60d","Type":"ContainerDied","Data":"3de89cc96d62d8cd20e4600ed10eae8e3d9f5d0069995438719efe52c28a76b9"} Jul 24 15:44:09.813879 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:09.813830 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" Jul 24 15:44:09.905807 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:09.905765 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8121c744-662a-4669-a7f0-9957f241f60d-openshift-service-ca\") pod \"8121c744-662a-4669-a7f0-9957f241f60d\" (UID: \"8121c744-662a-4669-a7f0-9957f241f60d\") " Jul 24 15:44:09.906053 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:09.905841 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9z2v\" (UniqueName: \"kubernetes.io/projected/8121c744-662a-4669-a7f0-9957f241f60d-kube-api-access-t9z2v\") pod \"8121c744-662a-4669-a7f0-9957f241f60d\" (UID: \"8121c744-662a-4669-a7f0-9957f241f60d\") " Jul 24 15:44:09.906053 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:09.905939 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8121c744-662a-4669-a7f0-9957f241f60d-maas-api-tls\") pod \"8121c744-662a-4669-a7f0-9957f241f60d\" (UID: \"8121c744-662a-4669-a7f0-9957f241f60d\") " Jul 24 15:44:09.906240 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:09.906210 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8121c744-662a-4669-a7f0-9957f241f60d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8121c744-662a-4669-a7f0-9957f241f60d" (UID: "8121c744-662a-4669-a7f0-9957f241f60d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:44:09.908371 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:09.908341 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8121c744-662a-4669-a7f0-9957f241f60d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8121c744-662a-4669-a7f0-9957f241f60d" (UID: "8121c744-662a-4669-a7f0-9957f241f60d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:44:09.908477 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:09.908386 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8121c744-662a-4669-a7f0-9957f241f60d-kube-api-access-t9z2v" (OuterVolumeSpecName: "kube-api-access-t9z2v") pod "8121c744-662a-4669-a7f0-9957f241f60d" (UID: "8121c744-662a-4669-a7f0-9957f241f60d"). InnerVolumeSpecName "kube-api-access-t9z2v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:44:10.007737 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:10.007692 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9z2v\" (UniqueName: \"kubernetes.io/projected/8121c744-662a-4669-a7f0-9957f241f60d-kube-api-access-t9z2v\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:44:10.007737 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:10.007731 2531 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8121c744-662a-4669-a7f0-9957f241f60d-maas-api-tls\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:44:10.007737 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:10.007741 2531 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8121c744-662a-4669-a7f0-9957f241f60d-openshift-service-ca\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:44:10.740299 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:10.740267 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" Jul 24 15:44:10.740783 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:10.740306 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr" event={"ID":"8121c744-662a-4669-a7f0-9957f241f60d","Type":"ContainerDied","Data":"c9a614f4606fac146968c2cce2c081ea3d3e41760ac315fb94c319f0af0c808c"} Jul 24 15:44:10.740783 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:10.740368 2531 scope.go:117] "RemoveContainer" containerID="3de89cc96d62d8cd20e4600ed10eae8e3d9f5d0069995438719efe52c28a76b9" Jul 24 15:44:10.756485 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:10.756450 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr"] Jul 24 15:44:10.759406 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:10.759376 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-923368-5556c5565f-6v7jr"] Jul 24 15:44:10.829574 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:10.829540 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-923368-gxv4k"] Jul 24 15:44:10.832647 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:10.832616 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-923368-gxv4k"] Jul 24 15:44:10.977801 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:10.977764 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:11.209562 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:11.209528 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:12.286236 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:12.286200 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42fd2aef-5f3d-45e7-9b75-1aaf49e583a0" path="/var/lib/kubelet/pods/42fd2aef-5f3d-45e7-9b75-1aaf49e583a0/volumes" Jul 24 15:44:12.286653 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:12.286639 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8121c744-662a-4669-a7f0-9957f241f60d" path="/var/lib/kubelet/pods/8121c744-662a-4669-a7f0-9957f241f60d/volumes" Jul 24 15:44:13.961801 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:13.961761 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:44:17.812761 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:17.812728 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:17.914162 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:17.914127 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:18.105453 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:18.105350 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:18.311803 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:18.311763 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:21.788025 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:21.787982 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cd9c98-srjvz"] Jul 24 15:44:21.788662 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:21.788639 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8121c744-662a-4669-a7f0-9957f241f60d" containerName="maas-api" Jul 24 15:44:21.788753 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:21.788666 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="8121c744-662a-4669-a7f0-9957f241f60d" containerName="maas-api" Jul 24 15:44:21.788753 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:21.788687 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42fd2aef-5f3d-45e7-9b75-1aaf49e583a0" containerName="revoke-keys" Jul 24 15:44:21.788753 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:21.788695 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="42fd2aef-5f3d-45e7-9b75-1aaf49e583a0" containerName="revoke-keys" Jul 24 15:44:21.788946 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:21.788851 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="8121c744-662a-4669-a7f0-9957f241f60d" containerName="maas-api" Jul 24 15:44:21.788946 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:21.788890 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="42fd2aef-5f3d-45e7-9b75-1aaf49e583a0" containerName="revoke-keys" Jul 24 15:44:21.792257 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:21.792229 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cd9c98-srjvz" Jul 24 15:44:21.795218 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:21.794998 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q8grb\"" Jul 24 15:44:21.799004 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:21.798972 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cd9c98-srjvz"] Jul 24 15:44:21.927806 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:21.927757 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5ebd571-eaec-4bbb-8947-6714d24f8895-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-cd9c98-srjvz\" (UID: \"c5ebd571-eaec-4bbb-8947-6714d24f8895\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cd9c98-srjvz" Jul 24 15:44:22.029044 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:22.029001 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5ebd571-eaec-4bbb-8947-6714d24f8895-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-cd9c98-srjvz\" (UID: \"c5ebd571-eaec-4bbb-8947-6714d24f8895\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cd9c98-srjvz" Jul 24 15:44:22.029715 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:22.029686 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5ebd571-eaec-4bbb-8947-6714d24f8895-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-cd9c98-srjvz\" (UID: \"c5ebd571-eaec-4bbb-8947-6714d24f8895\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cd9c98-srjvz" Jul 24 15:44:22.105736 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:22.105649 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cd9c98-srjvz" Jul 24 15:44:22.254710 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:22.254677 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cd9c98-srjvz"] Jul 24 15:44:22.256671 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:44:22.256644 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5ebd571_eaec_4bbb_8947_6714d24f8895.slice/crio-fb837eccc22b9deb1eb5df863212601b5a622f25396f641a664c35c647146cfd WatchSource:0}: Error finding container fb837eccc22b9deb1eb5df863212601b5a622f25396f641a664c35c647146cfd: Status 404 returned error can't find the container with id fb837eccc22b9deb1eb5df863212601b5a622f25396f641a664c35c647146cfd Jul 24 15:44:22.821514 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:22.821418 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cd9c98-srjvz" event={"ID":"c5ebd571-eaec-4bbb-8947-6714d24f8895","Type":"ContainerStarted","Data":"da1dcd136664579c52530520cd53efe13882451acb6a4b18e68ccba6fe9edca1"} Jul 24 15:44:22.821514 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:22.821476 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cd9c98-srjvz" event={"ID":"c5ebd571-eaec-4bbb-8947-6714d24f8895","Type":"ContainerStarted","Data":"fb837eccc22b9deb1eb5df863212601b5a622f25396f641a664c35c647146cfd"} Jul 24 15:44:22.835218 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:22.835158 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cd9c98-srjvz" podStartSLOduration=1.83514179 podStartE2EDuration="1.83514179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:44:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:44:22.834072883 +0000 UTC m=+2895.064549566" watchObservedRunningTime="2026-07-24 15:44:22.83514179 +0000 UTC m=+2895.065618474" Jul 24 15:44:23.829486 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:23.829447 2531 generic.go:356] "Generic (PLEG): container finished" podID="c5ebd571-eaec-4bbb-8947-6714d24f8895" containerID="da1dcd136664579c52530520cd53efe13882451acb6a4b18e68ccba6fe9edca1" exitCode=0 Jul 24 15:44:23.829926 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:23.829520 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cd9c98-srjvz" event={"ID":"c5ebd571-eaec-4bbb-8947-6714d24f8895","Type":"ContainerDied","Data":"da1dcd136664579c52530520cd53efe13882451acb6a4b18e68ccba6fe9edca1"} Jul 24 15:44:24.986270 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:24.986239 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cd9c98-srjvz" Jul 24 15:44:25.058653 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:25.058613 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5ebd571-eaec-4bbb-8947-6714d24f8895-maas-api-service-ca\") pod \"c5ebd571-eaec-4bbb-8947-6714d24f8895\" (UID: \"c5ebd571-eaec-4bbb-8947-6714d24f8895\") " Jul 24 15:44:25.059010 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:25.058988 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5ebd571-eaec-4bbb-8947-6714d24f8895-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c5ebd571-eaec-4bbb-8947-6714d24f8895" (UID: "c5ebd571-eaec-4bbb-8947-6714d24f8895"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:44:25.159793 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:25.159704 2531 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5ebd571-eaec-4bbb-8947-6714d24f8895-maas-api-service-ca\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:44:25.842591 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:25.842554 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cd9c98-srjvz" event={"ID":"c5ebd571-eaec-4bbb-8947-6714d24f8895","Type":"ContainerDied","Data":"fb837eccc22b9deb1eb5df863212601b5a622f25396f641a664c35c647146cfd"} Jul 24 15:44:25.842591 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:25.842590 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cd9c98-srjvz" Jul 24 15:44:25.842833 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:25.842608 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb837eccc22b9deb1eb5df863212601b5a622f25396f641a664c35c647146cfd" Jul 24 15:44:38.962233 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:38.962193 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:44:42.109308 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:42.109267 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:42.170237 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:42.170199 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:42.271861 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:42.271821 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:46.877588 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:46.877545 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:46.883280 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:46.883247 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cd9c98-srjvz"] Jul 24 15:44:46.886233 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:46.886203 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cd9c98-srjvz"] Jul 24 15:44:47.380527 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:47.380488 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:48.279224 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:48.279189 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5ebd571-eaec-4bbb-8947-6714d24f8895" path="/var/lib/kubelet/pods/c5ebd571-eaec-4bbb-8947-6714d24f8895/volumes" Jul 24 15:44:52.339240 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:52.339198 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:44:53.654812 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:44:53.654769 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:45:00.139771 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:00.139718 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6b942c-29748465-lrvnq"] Jul 24 15:45:00.140642 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:00.140614 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5ebd571-eaec-4bbb-8947-6714d24f8895" containerName="revoke-keys" Jul 24 15:45:00.140843 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:00.140826 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5ebd571-eaec-4bbb-8947-6714d24f8895" containerName="revoke-keys" Jul 24 15:45:00.141232 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:00.141096 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5ebd571-eaec-4bbb-8947-6714d24f8895" containerName="revoke-keys" Jul 24 15:45:00.144607 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:00.144579 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-053e45-29748465-65j9c"] Jul 24 15:45:00.144744 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:00.144700 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6b942c-29748465-lrvnq" Jul 24 15:45:00.147285 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:00.147257 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q8grb\"" Jul 24 15:45:00.148148 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:00.148128 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d419ce-29748465-62fxm"] Jul 24 15:45:00.148256 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:00.148239 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-053e45-29748465-65j9c" Jul 24 15:45:00.152115 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:00.152086 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d419ce-29748465-62fxm" Jul 24 15:45:00.154532 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:00.154504 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6b942c-29748465-lrvnq"] Jul 24 15:45:00.159672 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:00.159643 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-053e45-29748465-65j9c"] Jul 24 15:45:00.163773 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:00.163747 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6b942c-29748465-lrvnq" Jul 24 15:45:00.164349 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:00.164322 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d419ce-29748465-62fxm"] Jul 24 15:45:00.174228 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:00.174201 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-053e45-29748465-65j9c" Jul 24 15:45:00.217365 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:00.217204 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d419ce-29748465-62fxm" Jul 24 15:45:00.349329 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:00.349297 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6b942c-29748465-lrvnq"] Jul 24 15:45:00.351601 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:45:00.351555 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a10454a_3638_4880_84d0_e13cf6c77c6a.slice/crio-3caa35ad55a9fb93332d97cb3ee485ba46e45c8767482b3e65828de578b2e7d2 WatchSource:0}: Error finding container 3caa35ad55a9fb93332d97cb3ee485ba46e45c8767482b3e65828de578b2e7d2: Status 404 returned error can't find the container with id 3caa35ad55a9fb93332d97cb3ee485ba46e45c8767482b3e65828de578b2e7d2 Jul 24 15:45:00.390732 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:45:00.390655 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod813f2db2_33da_4d40_9b7a_c9feeb28e1f3.slice/crio-f0de36e387bbe8b40dffdfeea0f1d7ad1ce337e155a9cf318d4c8c5aec00f134 WatchSource:0}: Error finding container f0de36e387bbe8b40dffdfeea0f1d7ad1ce337e155a9cf318d4c8c5aec00f134: Status 404 returned error can't find the container with id f0de36e387bbe8b40dffdfeea0f1d7ad1ce337e155a9cf318d4c8c5aec00f134 Jul 24 15:45:00.391301 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:00.391272 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-053e45-29748465-65j9c"] Jul 24 15:45:00.419101 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:00.419072 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d419ce-29748465-62fxm"] Jul 24 15:45:00.420914 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:45:00.420860 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96fa681e_d5f9_4cd1_a776_92b6c5fdc36e.slice/crio-83f8e536ea4a7980818943cedf320d31bcac8223303e4a1f754a68cdbc3f392a WatchSource:0}: Error finding container 83f8e536ea4a7980818943cedf320d31bcac8223303e4a1f754a68cdbc3f392a: Status 404 returned error can't find the container with id 83f8e536ea4a7980818943cedf320d31bcac8223303e4a1f754a68cdbc3f392a Jul 24 15:45:01.074563 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:01.074455 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6b942c-29748465-lrvnq" event={"ID":"9a10454a-3638-4880-84d0-e13cf6c77c6a","Type":"ContainerStarted","Data":"265a1d21a749b2b234509cef2008cf78289ba1dadf15d7c6c1e9f898bb08fa90"} Jul 24 15:45:01.074563 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:01.074521 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6b942c-29748465-lrvnq" event={"ID":"9a10454a-3638-4880-84d0-e13cf6c77c6a","Type":"ContainerStarted","Data":"3caa35ad55a9fb93332d97cb3ee485ba46e45c8767482b3e65828de578b2e7d2"} Jul 24 15:45:01.076437 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:01.076404 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d419ce-29748465-62fxm" event={"ID":"96fa681e-d5f9-4cd1-a776-92b6c5fdc36e","Type":"ContainerStarted","Data":"e660a01716edd070c3845bcc4b0fb6517b4c88481f989ccc2fbe84c41c8d55d4"} Jul 24 15:45:01.076604 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:01.076469 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d419ce-29748465-62fxm" event={"ID":"96fa681e-d5f9-4cd1-a776-92b6c5fdc36e","Type":"ContainerStarted","Data":"83f8e536ea4a7980818943cedf320d31bcac8223303e4a1f754a68cdbc3f392a"} Jul 24 15:45:01.078178 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:01.078150 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-053e45-29748465-65j9c" event={"ID":"813f2db2-33da-4d40-9b7a-c9feeb28e1f3","Type":"ContainerStarted","Data":"ea3f4f6cb557bcc910fa9c0ce76aa0818e75888bc703143ee8df23852f5003f9"} Jul 24 15:45:01.078178 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:01.078196 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-053e45-29748465-65j9c" event={"ID":"813f2db2-33da-4d40-9b7a-c9feeb28e1f3","Type":"ContainerStarted","Data":"f0de36e387bbe8b40dffdfeea0f1d7ad1ce337e155a9cf318d4c8c5aec00f134"} Jul 24 15:45:01.091733 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:01.091677 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6b942c-29748465-lrvnq" podStartSLOduration=1.0916595550000001 podStartE2EDuration="1.091659555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:45:01.088516104 +0000 UTC m=+2933.318992779" watchObservedRunningTime="2026-07-24 15:45:01.091659555 +0000 UTC m=+2933.322136283" Jul 24 15:45:01.102626 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:01.102568 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d419ce-29748465-62fxm" podStartSLOduration=1.102549697 podStartE2EDuration="1.102549697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:45:01.101500829 +0000 UTC m=+2933.331977507" watchObservedRunningTime="2026-07-24 15:45:01.102549697 +0000 UTC m=+2933.333026382" Jul 24 15:45:02.086035 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:02.085995 2531 generic.go:356] "Generic (PLEG): container finished" podID="9a10454a-3638-4880-84d0-e13cf6c77c6a" containerID="265a1d21a749b2b234509cef2008cf78289ba1dadf15d7c6c1e9f898bb08fa90" exitCode=0 Jul 24 15:45:02.086517 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:02.086047 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6b942c-29748465-lrvnq" event={"ID":"9a10454a-3638-4880-84d0-e13cf6c77c6a","Type":"ContainerDied","Data":"265a1d21a749b2b234509cef2008cf78289ba1dadf15d7c6c1e9f898bb08fa90"} Jul 24 15:45:02.087825 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:02.087791 2531 generic.go:356] "Generic (PLEG): container finished" podID="96fa681e-d5f9-4cd1-a776-92b6c5fdc36e" containerID="e660a01716edd070c3845bcc4b0fb6517b4c88481f989ccc2fbe84c41c8d55d4" exitCode=0 Jul 24 15:45:02.087956 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:02.087843 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d419ce-29748465-62fxm" event={"ID":"96fa681e-d5f9-4cd1-a776-92b6c5fdc36e","Type":"ContainerDied","Data":"e660a01716edd070c3845bcc4b0fb6517b4c88481f989ccc2fbe84c41c8d55d4"} Jul 24 15:45:02.089426 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:02.089404 2531 generic.go:356] "Generic (PLEG): container finished" podID="813f2db2-33da-4d40-9b7a-c9feeb28e1f3" containerID="ea3f4f6cb557bcc910fa9c0ce76aa0818e75888bc703143ee8df23852f5003f9" exitCode=0 Jul 24 15:45:02.089516 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:02.089450 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-053e45-29748465-65j9c" event={"ID":"813f2db2-33da-4d40-9b7a-c9feeb28e1f3","Type":"ContainerDied","Data":"ea3f4f6cb557bcc910fa9c0ce76aa0818e75888bc703143ee8df23852f5003f9"} Jul 24 15:45:02.100577 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:02.100524 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-053e45-29748465-65j9c" podStartSLOduration=2.100508922 podStartE2EDuration="2.100508922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:45:01.115546859 +0000 UTC m=+2933.346023543" watchObservedRunningTime="2026-07-24 15:45:02.100508922 +0000 UTC m=+2934.330985606" Jul 24 15:45:03.261855 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:03.261821 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d419ce-29748465-62fxm" Jul 24 15:45:03.317841 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:03.317810 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-053e45-29748465-65j9c" Jul 24 15:45:03.321234 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:03.321199 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6b942c-29748465-lrvnq" Jul 24 15:45:04.104075 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:04.104042 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d419ce-29748465-62fxm" Jul 24 15:45:04.104235 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:04.104082 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d419ce-29748465-62fxm" event={"ID":"96fa681e-d5f9-4cd1-a776-92b6c5fdc36e","Type":"ContainerDied","Data":"83f8e536ea4a7980818943cedf320d31bcac8223303e4a1f754a68cdbc3f392a"} Jul 24 15:45:04.104235 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:04.104131 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="83f8e536ea4a7980818943cedf320d31bcac8223303e4a1f754a68cdbc3f392a" Jul 24 15:45:04.105537 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:04.105514 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-053e45-29748465-65j9c" Jul 24 15:45:04.105658 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:04.105530 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-053e45-29748465-65j9c" event={"ID":"813f2db2-33da-4d40-9b7a-c9feeb28e1f3","Type":"ContainerDied","Data":"f0de36e387bbe8b40dffdfeea0f1d7ad1ce337e155a9cf318d4c8c5aec00f134"} Jul 24 15:45:04.105658 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:04.105585 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0de36e387bbe8b40dffdfeea0f1d7ad1ce337e155a9cf318d4c8c5aec00f134" Jul 24 15:45:04.106990 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:04.106966 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6b942c-29748465-lrvnq" event={"ID":"9a10454a-3638-4880-84d0-e13cf6c77c6a","Type":"ContainerDied","Data":"3caa35ad55a9fb93332d97cb3ee485ba46e45c8767482b3e65828de578b2e7d2"} Jul 24 15:45:04.107114 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:04.106993 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6b942c-29748465-lrvnq" Jul 24 15:45:04.107114 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:04.107005 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3caa35ad55a9fb93332d97cb3ee485ba46e45c8767482b3e65828de578b2e7d2" Jul 24 15:45:24.972338 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:24.972301 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:45:29.080050 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:29.080015 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:45:29.827921 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:29.827878 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:45:35.949335 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:35.949292 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:45:50.556330 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:50.556290 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:45:50.658153 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:50.658118 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:45:50.844215 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:50.844125 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:45:51.078507 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:51.078469 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:45:51.295110 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:51.295070 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-053e45-6cj5m"] Jul 24 15:45:51.295527 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:51.295514 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="813f2db2-33da-4d40-9b7a-c9feeb28e1f3" containerName="cleanup" Jul 24 15:45:51.295578 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:51.295528 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="813f2db2-33da-4d40-9b7a-c9feeb28e1f3" containerName="cleanup" Jul 24 15:45:51.295578 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:51.295546 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a10454a-3638-4880-84d0-e13cf6c77c6a" containerName="cleanup" Jul 24 15:45:51.295578 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:51.295554 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a10454a-3638-4880-84d0-e13cf6c77c6a" containerName="cleanup" Jul 24 15:45:51.295578 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:51.295576 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96fa681e-d5f9-4cd1-a776-92b6c5fdc36e" containerName="cleanup" Jul 24 15:45:51.295578 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:51.295581 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="96fa681e-d5f9-4cd1-a776-92b6c5fdc36e" containerName="cleanup" Jul 24 15:45:51.295746 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:51.295653 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="813f2db2-33da-4d40-9b7a-c9feeb28e1f3" containerName="cleanup" Jul 24 15:45:51.295746 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:51.295660 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="96fa681e-d5f9-4cd1-a776-92b6c5fdc36e" containerName="cleanup" Jul 24 15:45:51.295746 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:51.295666 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a10454a-3638-4880-84d0-e13cf6c77c6a" containerName="cleanup" Jul 24 15:45:51.298998 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:51.298978 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-053e45-6cj5m" Jul 24 15:45:51.301214 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:51.301188 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q8grb\"" Jul 24 15:45:51.306083 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:51.306053 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-053e45-6cj5m"] Jul 24 15:45:51.453130 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:51.453068 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa5a05c2-b528-4da4-b203-87a11fcc5d49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-053e45-6cj5m\" (UID: \"fa5a05c2-b528-4da4-b203-87a11fcc5d49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-053e45-6cj5m" Jul 24 15:45:51.554349 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:51.554235 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa5a05c2-b528-4da4-b203-87a11fcc5d49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-053e45-6cj5m\" (UID: \"fa5a05c2-b528-4da4-b203-87a11fcc5d49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-053e45-6cj5m" Jul 24 15:45:51.554985 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:51.554957 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa5a05c2-b528-4da4-b203-87a11fcc5d49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-053e45-6cj5m\" (UID: \"fa5a05c2-b528-4da4-b203-87a11fcc5d49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-053e45-6cj5m" Jul 24 15:45:51.612538 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:51.612493 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-053e45-6cj5m" Jul 24 15:45:51.765016 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:51.764986 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-053e45-6cj5m"] Jul 24 15:45:51.768209 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:45:51.768167 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa5a05c2_b528_4da4_b203_87a11fcc5d49.slice/crio-e88c294fea5b73815d138230be1c60f7807b073b7eb08f96be367fb14b59eef3 WatchSource:0}: Error finding container e88c294fea5b73815d138230be1c60f7807b073b7eb08f96be367fb14b59eef3: Status 404 returned error can't find the container with id e88c294fea5b73815d138230be1c60f7807b073b7eb08f96be367fb14b59eef3 Jul 24 15:45:52.427955 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:52.427921 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-053e45-6cj5m" event={"ID":"fa5a05c2-b528-4da4-b203-87a11fcc5d49","Type":"ContainerStarted","Data":"ffcf9b12dc947e686df492b3b315090d1d43c100bc09c4c21635e183de97fead"} Jul 24 15:45:52.428144 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:52.427976 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-053e45-6cj5m" event={"ID":"fa5a05c2-b528-4da4-b203-87a11fcc5d49","Type":"ContainerStarted","Data":"e88c294fea5b73815d138230be1c60f7807b073b7eb08f96be367fb14b59eef3"} Jul 24 15:45:52.442849 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:52.442791 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-053e45-6cj5m" podStartSLOduration=1.442773795 podStartE2EDuration="1.442773795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:45:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:45:52.441773996 +0000 UTC m=+2984.672250680" watchObservedRunningTime="2026-07-24 15:45:52.442773795 +0000 UTC m=+2984.673250478" Jul 24 15:45:53.436831 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:53.436796 2531 generic.go:356] "Generic (PLEG): container finished" podID="fa5a05c2-b528-4da4-b203-87a11fcc5d49" containerID="ffcf9b12dc947e686df492b3b315090d1d43c100bc09c4c21635e183de97fead" exitCode=0 Jul 24 15:45:53.437247 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:53.436897 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-053e45-6cj5m" event={"ID":"fa5a05c2-b528-4da4-b203-87a11fcc5d49","Type":"ContainerDied","Data":"ffcf9b12dc947e686df492b3b315090d1d43c100bc09c4c21635e183de97fead"} Jul 24 15:45:54.594715 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:54.594680 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-053e45-6cj5m" Jul 24 15:45:54.685221 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:54.685190 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa5a05c2-b528-4da4-b203-87a11fcc5d49-maas-api-service-ca\") pod \"fa5a05c2-b528-4da4-b203-87a11fcc5d49\" (UID: \"fa5a05c2-b528-4da4-b203-87a11fcc5d49\") " Jul 24 15:45:54.685650 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:54.685623 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa5a05c2-b528-4da4-b203-87a11fcc5d49-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fa5a05c2-b528-4da4-b203-87a11fcc5d49" (UID: "fa5a05c2-b528-4da4-b203-87a11fcc5d49"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:45:54.786896 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:54.786761 2531 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa5a05c2-b528-4da4-b203-87a11fcc5d49-maas-api-service-ca\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:45:55.452126 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:55.452099 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-053e45-6cj5m" Jul 24 15:45:55.452309 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:55.452095 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-053e45-6cj5m" event={"ID":"fa5a05c2-b528-4da4-b203-87a11fcc5d49","Type":"ContainerDied","Data":"e88c294fea5b73815d138230be1c60f7807b073b7eb08f96be367fb14b59eef3"} Jul 24 15:45:55.452309 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:45:55.452219 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e88c294fea5b73815d138230be1c60f7807b073b7eb08f96be367fb14b59eef3" Jul 24 15:46:03.894917 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:03.894876 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968"] Jul 24 15:46:03.895480 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:03.895217 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" podUID="c7d84236-d9b8-4793-a83a-810a04c016db" containerName="maas-api" containerID="cri-o://7c7aa4298f7e8e4971fc65549630beef97e1342c13615dad95db5ff50773b3e4" gracePeriod=30 Jul 24 15:46:03.918758 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:03.918726 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:03.924996 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:03.924946 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-053e45-29748465-65j9c"] Jul 24 15:46:03.928134 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:03.928099 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-053e45-29748465-65j9c"] Jul 24 15:46:03.993304 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:03.993259 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:04.055040 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.055004 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:04.169419 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.169388 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" Jul 24 15:46:04.279043 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.279007 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="813f2db2-33da-4d40-9b7a-c9feeb28e1f3" path="/var/lib/kubelet/pods/813f2db2-33da-4d40-9b7a-c9feeb28e1f3/volumes" Jul 24 15:46:04.280936 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.280909 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8q89d\" (UniqueName: \"kubernetes.io/projected/c7d84236-d9b8-4793-a83a-810a04c016db-kube-api-access-8q89d\") pod \"c7d84236-d9b8-4793-a83a-810a04c016db\" (UID: \"c7d84236-d9b8-4793-a83a-810a04c016db\") " Jul 24 15:46:04.281053 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.280983 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7d84236-d9b8-4793-a83a-810a04c016db-openshift-service-ca\") pod \"c7d84236-d9b8-4793-a83a-810a04c016db\" (UID: \"c7d84236-d9b8-4793-a83a-810a04c016db\") " Jul 24 15:46:04.281123 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.281111 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7d84236-d9b8-4793-a83a-810a04c016db-maas-api-tls\") pod \"c7d84236-d9b8-4793-a83a-810a04c016db\" (UID: \"c7d84236-d9b8-4793-a83a-810a04c016db\") " Jul 24 15:46:04.281403 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.281378 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7d84236-d9b8-4793-a83a-810a04c016db-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c7d84236-d9b8-4793-a83a-810a04c016db" (UID: "c7d84236-d9b8-4793-a83a-810a04c016db"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:46:04.281537 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.281486 2531 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7d84236-d9b8-4793-a83a-810a04c016db-openshift-service-ca\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:46:04.283808 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.283769 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7d84236-d9b8-4793-a83a-810a04c016db-kube-api-access-8q89d" (OuterVolumeSpecName: "kube-api-access-8q89d") pod "c7d84236-d9b8-4793-a83a-810a04c016db" (UID: "c7d84236-d9b8-4793-a83a-810a04c016db"). InnerVolumeSpecName "kube-api-access-8q89d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:46:04.283988 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.283880 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7d84236-d9b8-4793-a83a-810a04c016db-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c7d84236-d9b8-4793-a83a-810a04c016db" (UID: "c7d84236-d9b8-4793-a83a-810a04c016db"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 15:46:04.383023 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.382974 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8q89d\" (UniqueName: \"kubernetes.io/projected/c7d84236-d9b8-4793-a83a-810a04c016db-kube-api-access-8q89d\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:46:04.383023 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.383011 2531 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7d84236-d9b8-4793-a83a-810a04c016db-maas-api-tls\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:46:04.516836 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.516796 2531 generic.go:356] "Generic (PLEG): container finished" podID="c7d84236-d9b8-4793-a83a-810a04c016db" containerID="7c7aa4298f7e8e4971fc65549630beef97e1342c13615dad95db5ff50773b3e4" exitCode=0 Jul 24 15:46:04.517040 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.516848 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" Jul 24 15:46:04.517040 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.516859 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" event={"ID":"c7d84236-d9b8-4793-a83a-810a04c016db","Type":"ContainerDied","Data":"7c7aa4298f7e8e4971fc65549630beef97e1342c13615dad95db5ff50773b3e4"} Jul 24 15:46:04.517040 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.516931 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968" event={"ID":"c7d84236-d9b8-4793-a83a-810a04c016db","Type":"ContainerDied","Data":"7d944eb9ecba526b9b7cc3dd4a4b5dde88ad405728bd3cf44885930496ea7c7b"} Jul 24 15:46:04.517040 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.516961 2531 scope.go:117] "RemoveContainer" containerID="7c7aa4298f7e8e4971fc65549630beef97e1342c13615dad95db5ff50773b3e4" Jul 24 15:46:04.528280 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.528251 2531 scope.go:117] "RemoveContainer" containerID="7c7aa4298f7e8e4971fc65549630beef97e1342c13615dad95db5ff50773b3e4" Jul 24 15:46:04.528731 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:46:04.528709 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c7aa4298f7e8e4971fc65549630beef97e1342c13615dad95db5ff50773b3e4\": container with ID starting with 7c7aa4298f7e8e4971fc65549630beef97e1342c13615dad95db5ff50773b3e4 not found: ID does not exist" containerID="7c7aa4298f7e8e4971fc65549630beef97e1342c13615dad95db5ff50773b3e4" Jul 24 15:46:04.528803 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.528742 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c7aa4298f7e8e4971fc65549630beef97e1342c13615dad95db5ff50773b3e4"} err="failed to get container status \"7c7aa4298f7e8e4971fc65549630beef97e1342c13615dad95db5ff50773b3e4\": rpc error: code = NotFound desc = could not find container \"7c7aa4298f7e8e4971fc65549630beef97e1342c13615dad95db5ff50773b3e4\": container with ID starting with 7c7aa4298f7e8e4971fc65549630beef97e1342c13615dad95db5ff50773b3e4 not found: ID does not exist" Jul 24 15:46:04.539392 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.539358 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968"] Jul 24 15:46:04.543727 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:04.543689 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-053e45-6846f84978-4m968"] Jul 24 15:46:06.278882 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:06.278833 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7d84236-d9b8-4793-a83a-810a04c016db" path="/var/lib/kubelet/pods/c7d84236-d9b8-4793-a83a-810a04c016db/volumes" Jul 24 15:46:06.355566 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:06.355530 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:06.357492 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:06.357466 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-053e45-6cj5m"] Jul 24 15:46:06.360158 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:06.360136 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-053e45-6cj5m"] Jul 24 15:46:06.773360 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:06.773327 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:07.061894 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:07.061766 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:46:08.279904 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:08.279845 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa5a05c2-b528-4da4-b203-87a11fcc5d49" path="/var/lib/kubelet/pods/fa5a05c2-b528-4da4-b203-87a11fcc5d49/volumes" Jul 24 15:46:08.568311 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:08.568220 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:46:08.584283 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:08.584258 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:46:12.857418 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:12.857381 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:12.957156 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:12.957125 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:13.109123 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:13.109018 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c"] Jul 24 15:46:13.109301 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:13.109245 2531 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" podUID="edc17d5c-7b1e-4196-9594-0d984bb8a10f" containerName="istio-proxy" containerID="cri-o://7511d6706c7ab5c5e5f32ff9897bb92fc70f968b2917746536f8dff2f476ee9c" gracePeriod=30 Jul 24 15:46:13.131231 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:13.131192 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:13.367293 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:13.367198 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:15.889595 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:15.889548 2531 patch_prober.go:28] interesting pod/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 15:46:15.890046 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:15.889620 2531 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" podUID="edc17d5c-7b1e-4196-9594-0d984bb8a10f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 15:46:18.387626 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.387552 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:46:18.532035 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.531996 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h68fk\" (UniqueName: \"kubernetes.io/projected/edc17d5c-7b1e-4196-9594-0d984bb8a10f-kube-api-access-h68fk\") pod \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " Jul 24 15:46:18.532035 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.532038 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-credential-socket\") pod \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " Jul 24 15:46:18.532300 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.532080 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-podinfo\") pod \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " Jul 24 15:46:18.532300 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.532126 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-envoy\") pod \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " Jul 24 15:46:18.532300 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.532150 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-data\") pod \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " Jul 24 15:46:18.532300 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.532201 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-token\") pod \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " Jul 24 15:46:18.532300 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.532273 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-workload-socket\") pod \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " Jul 24 15:46:18.532573 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.532307 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-workload-certs\") pod \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " Jul 24 15:46:18.532573 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.532348 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "edc17d5c-7b1e-4196-9594-0d984bb8a10f" (UID: "edc17d5c-7b1e-4196-9594-0d984bb8a10f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:46:18.532573 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.532409 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istiod-ca-cert\") pod \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\" (UID: \"edc17d5c-7b1e-4196-9594-0d984bb8a10f\") " Jul 24 15:46:18.532573 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.532551 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "edc17d5c-7b1e-4196-9594-0d984bb8a10f" (UID: "edc17d5c-7b1e-4196-9594-0d984bb8a10f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:46:18.532573 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.532560 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "edc17d5c-7b1e-4196-9594-0d984bb8a10f" (UID: "edc17d5c-7b1e-4196-9594-0d984bb8a10f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:46:18.532899 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.532841 2531 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-workload-socket\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:46:18.533204 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.532933 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "edc17d5c-7b1e-4196-9594-0d984bb8a10f" (UID: "edc17d5c-7b1e-4196-9594-0d984bb8a10f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:46:18.533301 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.533223 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-data" (OuterVolumeSpecName: "istio-data") pod "edc17d5c-7b1e-4196-9594-0d984bb8a10f" (UID: "edc17d5c-7b1e-4196-9594-0d984bb8a10f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:46:18.533478 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.533442 2531 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-workload-certs\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:46:18.533549 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.533478 2531 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-credential-socket\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:46:18.534602 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.534578 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "edc17d5c-7b1e-4196-9594-0d984bb8a10f" (UID: "edc17d5c-7b1e-4196-9594-0d984bb8a10f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 15:46:18.534707 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.534685 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edc17d5c-7b1e-4196-9594-0d984bb8a10f-kube-api-access-h68fk" (OuterVolumeSpecName: "kube-api-access-h68fk") pod "edc17d5c-7b1e-4196-9594-0d984bb8a10f" (UID: "edc17d5c-7b1e-4196-9594-0d984bb8a10f"). InnerVolumeSpecName "kube-api-access-h68fk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:46:18.535083 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.535065 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "edc17d5c-7b1e-4196-9594-0d984bb8a10f" (UID: "edc17d5c-7b1e-4196-9594-0d984bb8a10f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 15:46:18.535169 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.535075 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-token" (OuterVolumeSpecName: "istio-token") pod "edc17d5c-7b1e-4196-9594-0d984bb8a10f" (UID: "edc17d5c-7b1e-4196-9594-0d984bb8a10f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 15:46:18.611119 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.611079 2531 generic.go:356] "Generic (PLEG): container finished" podID="edc17d5c-7b1e-4196-9594-0d984bb8a10f" containerID="7511d6706c7ab5c5e5f32ff9897bb92fc70f968b2917746536f8dff2f476ee9c" exitCode=0 Jul 24 15:46:18.611292 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.611137 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" Jul 24 15:46:18.611292 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.611171 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" event={"ID":"edc17d5c-7b1e-4196-9594-0d984bb8a10f","Type":"ContainerDied","Data":"7511d6706c7ab5c5e5f32ff9897bb92fc70f968b2917746536f8dff2f476ee9c"} Jul 24 15:46:18.611292 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.611237 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c" event={"ID":"edc17d5c-7b1e-4196-9594-0d984bb8a10f","Type":"ContainerDied","Data":"ba34b5c68dca49e57d60dc13d309cd4f27ff089183868fcaef361afcb43ea0fe"} Jul 24 15:46:18.611292 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.611270 2531 scope.go:117] "RemoveContainer" containerID="7511d6706c7ab5c5e5f32ff9897bb92fc70f968b2917746536f8dff2f476ee9c" Jul 24 15:46:18.622950 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.622927 2531 scope.go:117] "RemoveContainer" containerID="b9cea02b3ff3d0719088b835e7d4d68d2ff9f86c748cd192549cfff0a1991e3f" Jul 24 15:46:18.633021 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.632954 2531 scope.go:117] "RemoveContainer" containerID="7511d6706c7ab5c5e5f32ff9897bb92fc70f968b2917746536f8dff2f476ee9c" Jul 24 15:46:18.634111 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:46:18.633484 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7511d6706c7ab5c5e5f32ff9897bb92fc70f968b2917746536f8dff2f476ee9c\": container with ID starting with 7511d6706c7ab5c5e5f32ff9897bb92fc70f968b2917746536f8dff2f476ee9c not found: ID does not exist" containerID="7511d6706c7ab5c5e5f32ff9897bb92fc70f968b2917746536f8dff2f476ee9c" Jul 24 15:46:18.634111 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.633529 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7511d6706c7ab5c5e5f32ff9897bb92fc70f968b2917746536f8dff2f476ee9c"} err="failed to get container status \"7511d6706c7ab5c5e5f32ff9897bb92fc70f968b2917746536f8dff2f476ee9c\": rpc error: code = NotFound desc = could not find container \"7511d6706c7ab5c5e5f32ff9897bb92fc70f968b2917746536f8dff2f476ee9c\": container with ID starting with 7511d6706c7ab5c5e5f32ff9897bb92fc70f968b2917746536f8dff2f476ee9c not found: ID does not exist" Jul 24 15:46:18.634111 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.633559 2531 scope.go:117] "RemoveContainer" containerID="b9cea02b3ff3d0719088b835e7d4d68d2ff9f86c748cd192549cfff0a1991e3f" Jul 24 15:46:18.634406 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.634382 2531 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h68fk\" (UniqueName: \"kubernetes.io/projected/edc17d5c-7b1e-4196-9594-0d984bb8a10f-kube-api-access-h68fk\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:46:18.634471 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.634413 2531 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-podinfo\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:46:18.634471 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.634430 2531 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-envoy\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:46:18.634471 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.634446 2531 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-data\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:46:18.634471 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.634458 2531 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istio-token\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:46:18.634622 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.634472 2531 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edc17d5c-7b1e-4196-9594-0d984bb8a10f-istiod-ca-cert\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:46:18.634997 ip-10-0-137-108 kubenswrapper[2531]: E0724 15:46:18.634969 2531 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9cea02b3ff3d0719088b835e7d4d68d2ff9f86c748cd192549cfff0a1991e3f\": container with ID starting with b9cea02b3ff3d0719088b835e7d4d68d2ff9f86c748cd192549cfff0a1991e3f not found: ID does not exist" containerID="b9cea02b3ff3d0719088b835e7d4d68d2ff9f86c748cd192549cfff0a1991e3f" Jul 24 15:46:18.635100 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.635007 2531 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9cea02b3ff3d0719088b835e7d4d68d2ff9f86c748cd192549cfff0a1991e3f"} err="failed to get container status \"b9cea02b3ff3d0719088b835e7d4d68d2ff9f86c748cd192549cfff0a1991e3f\": rpc error: code = NotFound desc = could not find container \"b9cea02b3ff3d0719088b835e7d4d68d2ff9f86c748cd192549cfff0a1991e3f\": container with ID starting with b9cea02b3ff3d0719088b835e7d4d68d2ff9f86c748cd192549cfff0a1991e3f not found: ID does not exist" Jul 24 15:46:18.636716 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.636694 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c"] Jul 24 15:46:18.642261 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:18.642201 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-053e45-openshift-default-649cc4bbcb-whp5c"] Jul 24 15:46:20.278992 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:20.278952 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edc17d5c-7b1e-4196-9594-0d984bb8a10f" path="/var/lib/kubelet/pods/edc17d5c-7b1e-4196-9594-0d984bb8a10f/volumes" Jul 24 15:46:25.742355 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:25.742308 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d419ce-29748465-62fxm"] Jul 24 15:46:25.747072 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:25.747029 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:25.749019 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:25.748994 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-d419ce-29748465-62fxm"] Jul 24 15:46:25.827180 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:25.827145 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:25.894116 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:25.894078 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:26.277925 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:26.277857 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96fa681e-d5f9-4cd1-a776-92b6c5fdc36e" path="/var/lib/kubelet/pods/96fa681e-d5f9-4cd1-a776-92b6c5fdc36e/volumes" Jul 24 15:46:28.678015 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:28.677974 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:29.082279 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:29.082183 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:29.266107 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:29.266062 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:46:35.307653 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:35.307615 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:35.405432 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:35.405395 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:35.600308 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:35.600220 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:35.818042 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:35.818007 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:35.937312 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:35.937265 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6b942c-8hvzv"] Jul 24 15:46:35.937857 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:35.937826 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edc17d5c-7b1e-4196-9594-0d984bb8a10f" containerName="istio-proxy" Jul 24 15:46:35.937857 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:35.937857 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="edc17d5c-7b1e-4196-9594-0d984bb8a10f" containerName="istio-proxy" Jul 24 15:46:35.938050 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:35.937892 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7d84236-d9b8-4793-a83a-810a04c016db" containerName="maas-api" Jul 24 15:46:35.938050 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:35.937901 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d84236-d9b8-4793-a83a-810a04c016db" containerName="maas-api" Jul 24 15:46:35.938050 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:35.937955 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa5a05c2-b528-4da4-b203-87a11fcc5d49" containerName="revoke-keys" Jul 24 15:46:35.938050 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:35.937964 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa5a05c2-b528-4da4-b203-87a11fcc5d49" containerName="revoke-keys" Jul 24 15:46:35.938241 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:35.938067 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="edc17d5c-7b1e-4196-9594-0d984bb8a10f" containerName="istio-proxy" Jul 24 15:46:35.938241 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:35.938079 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="edc17d5c-7b1e-4196-9594-0d984bb8a10f" containerName="istio-proxy" Jul 24 15:46:35.938241 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:35.938091 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa5a05c2-b528-4da4-b203-87a11fcc5d49" containerName="revoke-keys" Jul 24 15:46:35.938241 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:35.938101 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7d84236-d9b8-4793-a83a-810a04c016db" containerName="maas-api" Jul 24 15:46:35.943114 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:35.943092 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6b942c-8hvzv" Jul 24 15:46:35.945531 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:35.945507 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-q8grb\"" Jul 24 15:46:35.948147 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:35.948121 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6b942c-8hvzv"] Jul 24 15:46:35.999080 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:35.999047 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72d6bace-0560-4efb-970b-178def73c811-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-6b942c-8hvzv\" (UID: \"72d6bace-0560-4efb-970b-178def73c811\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6b942c-8hvzv" Jul 24 15:46:36.099940 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:36.099902 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72d6bace-0560-4efb-970b-178def73c811-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-6b942c-8hvzv\" (UID: \"72d6bace-0560-4efb-970b-178def73c811\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6b942c-8hvzv" Jul 24 15:46:36.100520 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:36.100498 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72d6bace-0560-4efb-970b-178def73c811-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-6b942c-8hvzv\" (UID: \"72d6bace-0560-4efb-970b-178def73c811\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6b942c-8hvzv" Jul 24 15:46:36.256361 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:36.256258 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6b942c-8hvzv" Jul 24 15:46:36.410289 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:36.410262 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6b942c-8hvzv"] Jul 24 15:46:36.411608 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:46:36.411581 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72d6bace_0560_4efb_970b_178def73c811.slice/crio-477e7d63fdd4ef2604f6371fcd1bbabf4c414bf2cb08005e8167d4b1a467aa04 WatchSource:0}: Error finding container 477e7d63fdd4ef2604f6371fcd1bbabf4c414bf2cb08005e8167d4b1a467aa04: Status 404 returned error can't find the container with id 477e7d63fdd4ef2604f6371fcd1bbabf4c414bf2cb08005e8167d4b1a467aa04 Jul 24 15:46:36.730286 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:36.730249 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6b942c-8hvzv" event={"ID":"72d6bace-0560-4efb-970b-178def73c811","Type":"ContainerStarted","Data":"533f48a2340c454da848539c867ddbe870f40a242bcf8210ab7565c7de8ab0eb"} Jul 24 15:46:36.730479 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:36.730314 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6b942c-8hvzv" event={"ID":"72d6bace-0560-4efb-970b-178def73c811","Type":"ContainerStarted","Data":"477e7d63fdd4ef2604f6371fcd1bbabf4c414bf2cb08005e8167d4b1a467aa04"} Jul 24 15:46:36.747072 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:36.747013 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6b942c-8hvzv" podStartSLOduration=1.7469971210000002 podStartE2EDuration="1.746997121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:46:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:46:36.744564575 +0000 UTC m=+3028.975041290" watchObservedRunningTime="2026-07-24 15:46:36.746997121 +0000 UTC m=+3028.977473804" Jul 24 15:46:37.739595 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:37.739558 2531 generic.go:356] "Generic (PLEG): container finished" podID="72d6bace-0560-4efb-970b-178def73c811" containerID="533f48a2340c454da848539c867ddbe870f40a242bcf8210ab7565c7de8ab0eb" exitCode=0 Jul 24 15:46:37.740027 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:37.739639 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6b942c-8hvzv" event={"ID":"72d6bace-0560-4efb-970b-178def73c811","Type":"ContainerDied","Data":"533f48a2340c454da848539c867ddbe870f40a242bcf8210ab7565c7de8ab0eb"} Jul 24 15:46:38.889263 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:38.889229 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6b942c-8hvzv" Jul 24 15:46:38.927392 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:38.927361 2531 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72d6bace-0560-4efb-970b-178def73c811-maas-api-service-ca\") pod \"72d6bace-0560-4efb-970b-178def73c811\" (UID: \"72d6bace-0560-4efb-970b-178def73c811\") " Jul 24 15:46:38.927742 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:38.927715 2531 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72d6bace-0560-4efb-970b-178def73c811-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "72d6bace-0560-4efb-970b-178def73c811" (UID: "72d6bace-0560-4efb-970b-178def73c811"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 15:46:39.028943 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:39.028838 2531 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72d6bace-0560-4efb-970b-178def73c811-maas-api-service-ca\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 24 15:46:39.754583 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:39.754549 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6b942c-8hvzv" event={"ID":"72d6bace-0560-4efb-970b-178def73c811","Type":"ContainerDied","Data":"477e7d63fdd4ef2604f6371fcd1bbabf4c414bf2cb08005e8167d4b1a467aa04"} Jul 24 15:46:39.754748 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:39.754580 2531 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6b942c-8hvzv" Jul 24 15:46:39.754748 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:39.754607 2531 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="477e7d63fdd4ef2604f6371fcd1bbabf4c414bf2cb08005e8167d4b1a467aa04" Jul 24 15:46:39.975318 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:39.975281 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:44.964968 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:44.964928 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:46.323327 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:46.323285 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:46.324822 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:46.324793 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6b942c-29748465-lrvnq"] Jul 24 15:46:46.330031 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:46.330004 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-6b942c-29748465-lrvnq"] Jul 24 15:46:46.480059 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:46.480024 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:46.580967 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:46.580856 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:48.278049 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:48.278013 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a10454a-3638-4880-84d0-e13cf6c77c6a" path="/var/lib/kubelet/pods/9a10454a-3638-4880-84d0-e13cf6c77c6a/volumes" Jul 24 15:46:49.356646 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:49.356604 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-ljfxb"] Jul 24 15:46:51.014373 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:51.014340 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:51.030678 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:51.030647 2531 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6b942c-8hvzv"] Jul 24 15:46:51.033297 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:51.033275 2531 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6b942c-8hvzv"] Jul 24 15:46:51.412323 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:51.412234 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:52.278001 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:52.277967 2531 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72d6bace-0560-4efb-970b-178def73c811" path="/var/lib/kubelet/pods/72d6bace-0560-4efb-970b-178def73c811/volumes" Jul 24 15:46:57.671456 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:57.671424 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:57.745613 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:57.745577 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:57.975531 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:57.975438 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:58.207198 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:58.207158 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:46:58.615297 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:46:58.615258 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6954746ddf-mgsft_de5f106e-828b-4754-8440-61fa371ddc71/authorino/0.log" Jul 24 15:47:04.199422 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:04.199334 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-ng4q9_1d69f3f6-a0cc-45aa-8d2d-cc0b4cf5017b/manager/0.log" Jul 24 15:47:04.339573 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:04.339537 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-cs6dm_6060f538-1807-499a-b224-e07dffbebbe8/manager/0.log" Jul 24 15:47:04.622057 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:04.621981 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-zk72b_541aed6a-215c-465a-b8ff-dfcd57c9f107/server/0.log" Jul 24 15:47:04.881636 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:04.881560 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f5584866b-6mp2c_077904ac-fa0f-4a6c-b19e-9e9b6da1b68f/copy-manifests/0.log" Jul 24 15:47:04.891581 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:04.891552 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f5584866b-6mp2c_077904ac-fa0f-4a6c-b19e-9e9b6da1b68f/manager/0.log" Jul 24 15:47:05.549540 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:05.549512 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-64775d5d55-9xvvz_e4f641a2-ee07-4901-80cd-ab54a50faebf/maas-api/0.log" Jul 24 15:47:06.823256 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:06.823226 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm_cf08aa03-1a28-4150-9d2e-54726ea50f51/util/0.log" Jul 24 15:47:06.831918 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:06.831892 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm_cf08aa03-1a28-4150-9d2e-54726ea50f51/pull/0.log" Jul 24 15:47:06.839378 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:06.839351 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm_cf08aa03-1a28-4150-9d2e-54726ea50f51/extract/0.log" Jul 24 15:47:06.963911 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:06.963861 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7_b653e476-5ebd-4c56-beb9-8ed2aaa66c42/util/0.log" Jul 24 15:47:06.973971 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:06.973948 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7_b653e476-5ebd-4c56-beb9-8ed2aaa66c42/pull/0.log" Jul 24 15:47:06.987305 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:06.987283 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7_b653e476-5ebd-4c56-beb9-8ed2aaa66c42/extract/0.log" Jul 24 15:47:07.116538 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:07.116458 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv_f19e65a9-2996-41cd-89ad-80a78d4adda2/util/0.log" Jul 24 15:47:07.128780 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:07.128755 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv_f19e65a9-2996-41cd-89ad-80a78d4adda2/pull/0.log" Jul 24 15:47:07.139856 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:07.139825 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv_f19e65a9-2996-41cd-89ad-80a78d4adda2/extract/0.log" Jul 24 15:47:07.267211 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:07.267186 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz_8034980c-866f-4c7e-9d54-17f904645c0e/util/0.log" Jul 24 15:47:07.290500 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:07.290473 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz_8034980c-866f-4c7e-9d54-17f904645c0e/pull/0.log" Jul 24 15:47:07.300730 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:07.300708 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz_8034980c-866f-4c7e-9d54-17f904645c0e/extract/0.log" Jul 24 15:47:07.422290 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:07.422262 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6954746ddf-mgsft_de5f106e-828b-4754-8440-61fa371ddc71/authorino/0.log" Jul 24 15:47:07.563292 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:07.563261 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-s5kck_3504103a-4dcf-400b-a0a5-578d2c374fd2/manager/0.log" Jul 24 15:47:07.691275 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:07.691194 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-kx664_5a357ffd-7231-4fb2-8fec-3b050c5a64bc/manager/0.log" Jul 24 15:47:07.954925 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:07.954828 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xlk25_701e2e43-0ad1-451c-8491-ebf9b2ca9648/registry-server/0.log" Jul 24 15:47:08.212771 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:08.212690 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-ljfxb_34a9573a-009d-41e7-802f-4578118e3d02/limitador/0.log" Jul 24 15:47:08.336169 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:08.336135 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-sbbpv_094e5e26-87ea-4c9a-a49d-b44b9368b2bd/manager/0.log" Jul 24 15:47:09.082083 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:09.081996 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-758f749c7-9xg27_f32962bc-3bd3-41f4-89fe-245a37e1582f/kube-auth-proxy/0.log" Jul 24 15:47:09.207447 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:09.207401 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-gzdst_0e873231-3e07-4d1d-9d0d-bf7cb190d9c3/istio-proxy/1.log" Jul 24 15:47:09.456789 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:09.456750 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-hfcxx_58138eae-76c2-489f-abdb-f858b5ef0108/payload-processing/0.log" Jul 24 15:47:09.580918 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:09.580887 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-776fc686df-t8g55_3ede17c9-5194-41b7-a4e6-55a258fe1a4e/router/0.log" Jul 24 15:47:09.958250 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:09.958217 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-hv88q_407c4619-b7cb-43d2-8ab9-d15fead0de5e/main/0.log" Jul 24 15:47:10.320777 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:10.320694 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f7qjxs_ed14cd07-72ec-4afa-a66b-11292da5ddb0/main/0.log" Jul 24 15:47:11.768109 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:11.768077 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:47:11.869602 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:11.869563 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:47:16.133196 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:16.133159 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:47:16.210777 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:16.210732 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:47:19.002703 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:19.002676 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6jf8n_441a2a3d-5df6-4a88-92cc-63d18ba4310f/global-pull-secret-syncer/0.log" Jul 24 15:47:19.151941 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:19.151910 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" Jul 24 15:47:19.276846 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:19.276762 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-108.ec2.internal_fd97724e3608d55c7f74ae763e45fb90/haproxy/0.log" Jul 24 15:47:19.712706 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:19.712669 2531 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fcmhs_a82a6065-127d-4ced-9487-59b5c2849dbc/konnectivity-agent/0.log" limit=1024 Jul 24 15:47:24.405504 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:24.405474 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm_cf08aa03-1a28-4150-9d2e-54726ea50f51/extract/0.log" Jul 24 15:47:24.431302 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:24.431265 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm_cf08aa03-1a28-4150-9d2e-54726ea50f51/util/0.log" Jul 24 15:47:24.456333 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:24.456300 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bsbbm_cf08aa03-1a28-4150-9d2e-54726ea50f51/pull/0.log" Jul 24 15:47:24.490119 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:24.490090 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7_b653e476-5ebd-4c56-beb9-8ed2aaa66c42/extract/0.log" Jul 24 15:47:24.514275 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:24.514218 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7_b653e476-5ebd-4c56-beb9-8ed2aaa66c42/util/0.log" Jul 24 15:47:24.540470 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:24.540440 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jqkn7_b653e476-5ebd-4c56-beb9-8ed2aaa66c42/pull/0.log" Jul 24 15:47:24.569182 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:24.569148 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv_f19e65a9-2996-41cd-89ad-80a78d4adda2/extract/0.log" Jul 24 15:47:24.593983 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:24.593954 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv_f19e65a9-2996-41cd-89ad-80a78d4adda2/util/0.log" Jul 24 15:47:24.641094 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:24.641057 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fcntv_f19e65a9-2996-41cd-89ad-80a78d4adda2/pull/0.log" Jul 24 15:47:24.671413 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:24.671385 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz_8034980c-866f-4c7e-9d54-17f904645c0e/extract/0.log" Jul 24 15:47:24.695612 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:24.695586 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz_8034980c-866f-4c7e-9d54-17f904645c0e/util/0.log" Jul 24 15:47:24.721899 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:24.721841 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15qdhz_8034980c-866f-4c7e-9d54-17f904645c0e/pull/0.log" Jul 24 15:47:25.107419 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:25.107323 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6954746ddf-mgsft_de5f106e-828b-4754-8440-61fa371ddc71/authorino/0.log" Jul 24 15:47:25.144336 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:25.144295 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-s5kck_3504103a-4dcf-400b-a0a5-578d2c374fd2/manager/0.log" Jul 24 15:47:25.176441 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:25.176407 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-kx664_5a357ffd-7231-4fb2-8fec-3b050c5a64bc/manager/0.log" Jul 24 15:47:25.271644 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:25.271615 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xlk25_701e2e43-0ad1-451c-8491-ebf9b2ca9648/registry-server/0.log" Jul 24 15:47:25.470663 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:25.470626 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-ljfxb_34a9573a-009d-41e7-802f-4578118e3d02/limitador/0.log" Jul 24 15:47:25.552695 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:25.552653 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-sbbpv_094e5e26-87ea-4c9a-a49d-b44b9368b2bd/manager/0.log" Jul 24 15:47:27.161478 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:27.161446 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_68002b52-264d-4260-8bbb-eebafa650c9d/alertmanager/0.log" Jul 24 15:47:27.210065 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:27.210038 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_68002b52-264d-4260-8bbb-eebafa650c9d/config-reloader/0.log" Jul 24 15:47:27.250555 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:27.250529 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_68002b52-264d-4260-8bbb-eebafa650c9d/kube-rbac-proxy-web/0.log" Jul 24 15:47:27.302419 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:27.302390 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_68002b52-264d-4260-8bbb-eebafa650c9d/kube-rbac-proxy/0.log" Jul 24 15:47:27.333012 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:27.332979 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_68002b52-264d-4260-8bbb-eebafa650c9d/kube-rbac-proxy-metric/0.log" Jul 24 15:47:27.366758 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:27.366734 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_68002b52-264d-4260-8bbb-eebafa650c9d/prom-label-proxy/0.log" Jul 24 15:47:27.400399 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:27.400369 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_68002b52-264d-4260-8bbb-eebafa650c9d/init-config-reloader/0.log" Jul 24 15:47:27.480395 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:27.480360 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-86bxn_2872026a-6cfb-4c62-8ae5-b33297a9b1e7/cluster-monitoring-operator/0.log" Jul 24 15:47:27.517598 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:27.517562 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-7ljrb_98a7d25e-e0dc-4170-b75c-826ac325a54b/kube-state-metrics/0.log" Jul 24 15:47:27.549451 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:27.549421 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-7ljrb_98a7d25e-e0dc-4170-b75c-826ac325a54b/kube-rbac-proxy-main/0.log" Jul 24 15:47:27.602446 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:27.602415 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-7ljrb_98a7d25e-e0dc-4170-b75c-826ac325a54b/kube-rbac-proxy-self/0.log" Jul 24 15:47:27.748182 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:27.748088 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-gc7xf_32e69dbd-71cf-4709-99e0-32c3db1d9a74/monitoring-plugin/0.log" Jul 24 15:47:27.839703 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:27.839670 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2d4gx_7fbb1524-c4a9-4478-b9ab-be53b6f81309/node-exporter/0.log" Jul 24 15:47:27.898753 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:27.898727 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2d4gx_7fbb1524-c4a9-4478-b9ab-be53b6f81309/kube-rbac-proxy/0.log" Jul 24 15:47:27.963283 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:27.963254 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2d4gx_7fbb1524-c4a9-4478-b9ab-be53b6f81309/init-textfile/0.log" Jul 24 15:47:28.241659 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:28.241625 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-llnsz_6550c456-f2c1-4d34-8812-e14246b4aae0/kube-rbac-proxy-main/0.log" Jul 24 15:47:28.275446 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:28.275415 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-llnsz_6550c456-f2c1-4d34-8812-e14246b4aae0/kube-rbac-proxy-self/0.log" Jul 24 15:47:28.311816 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:28.311781 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-llnsz_6550c456-f2c1-4d34-8812-e14246b4aae0/openshift-state-metrics/0.log" Jul 24 15:47:28.905474 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:28.905436 2531 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l"] Jul 24 15:47:28.905963 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:28.905945 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72d6bace-0560-4efb-970b-178def73c811" containerName="revoke-keys" Jul 24 15:47:28.906076 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:28.905965 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="72d6bace-0560-4efb-970b-178def73c811" containerName="revoke-keys" Jul 24 15:47:28.906076 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:28.906020 2531 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edc17d5c-7b1e-4196-9594-0d984bb8a10f" containerName="istio-proxy" Jul 24 15:47:28.906076 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:28.906026 2531 state_mem.go:107] "Deleted CPUSet assignment" podUID="edc17d5c-7b1e-4196-9594-0d984bb8a10f" containerName="istio-proxy" Jul 24 15:47:28.906237 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:28.906120 2531 memory_manager.go:368] "RemoveStaleState removing state" podUID="72d6bace-0560-4efb-970b-178def73c811" containerName="revoke-keys" Jul 24 15:47:28.909546 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:28.909524 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" Jul 24 15:47:28.914574 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:28.914553 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fd7vt\"/\"kube-root-ca.crt\"" Jul 24 15:47:28.914817 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:28.914799 2531 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fd7vt\"/\"openshift-service-ca.crt\"" Jul 24 15:47:28.915188 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:28.915175 2531 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fd7vt\"/\"default-dockercfg-sprr5\"" Jul 24 15:47:28.929785 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:28.929753 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l"] Jul 24 15:47:29.003308 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:29.003254 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/973ac0cb-cb79-47fd-af8b-b210556af70f-podres\") pod \"perf-node-gather-daemonset-g4z5l\" (UID: \"973ac0cb-cb79-47fd-af8b-b210556af70f\") " pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" Jul 24 15:47:29.003308 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:29.003306 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4gmn\" (UniqueName: \"kubernetes.io/projected/973ac0cb-cb79-47fd-af8b-b210556af70f-kube-api-access-j4gmn\") pod \"perf-node-gather-daemonset-g4z5l\" (UID: \"973ac0cb-cb79-47fd-af8b-b210556af70f\") " pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" Jul 24 15:47:29.003308 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:29.003326 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/973ac0cb-cb79-47fd-af8b-b210556af70f-proc\") pod \"perf-node-gather-daemonset-g4z5l\" (UID: \"973ac0cb-cb79-47fd-af8b-b210556af70f\") " pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" Jul 24 15:47:29.003574 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:29.003419 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/973ac0cb-cb79-47fd-af8b-b210556af70f-lib-modules\") pod \"perf-node-gather-daemonset-g4z5l\" (UID: \"973ac0cb-cb79-47fd-af8b-b210556af70f\") " pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" Jul 24 15:47:29.003574 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:29.003447 2531 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/973ac0cb-cb79-47fd-af8b-b210556af70f-sys\") pod \"perf-node-gather-daemonset-g4z5l\" (UID: \"973ac0cb-cb79-47fd-af8b-b210556af70f\") " pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" Jul 24 15:47:29.104582 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:29.104520 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/973ac0cb-cb79-47fd-af8b-b210556af70f-proc\") pod \"perf-node-gather-daemonset-g4z5l\" (UID: \"973ac0cb-cb79-47fd-af8b-b210556af70f\") " pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" Jul 24 15:47:29.104780 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:29.104622 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/973ac0cb-cb79-47fd-af8b-b210556af70f-lib-modules\") pod \"perf-node-gather-daemonset-g4z5l\" (UID: \"973ac0cb-cb79-47fd-af8b-b210556af70f\") " pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" Jul 24 15:47:29.104780 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:29.104649 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/973ac0cb-cb79-47fd-af8b-b210556af70f-sys\") pod \"perf-node-gather-daemonset-g4z5l\" (UID: \"973ac0cb-cb79-47fd-af8b-b210556af70f\") " pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" Jul 24 15:47:29.104780 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:29.104671 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/973ac0cb-cb79-47fd-af8b-b210556af70f-proc\") pod \"perf-node-gather-daemonset-g4z5l\" (UID: \"973ac0cb-cb79-47fd-af8b-b210556af70f\") " pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" Jul 24 15:47:29.104780 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:29.104730 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/973ac0cb-cb79-47fd-af8b-b210556af70f-sys\") pod \"perf-node-gather-daemonset-g4z5l\" (UID: \"973ac0cb-cb79-47fd-af8b-b210556af70f\") " pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" Jul 24 15:47:29.104973 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:29.104815 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/973ac0cb-cb79-47fd-af8b-b210556af70f-lib-modules\") pod \"perf-node-gather-daemonset-g4z5l\" (UID: \"973ac0cb-cb79-47fd-af8b-b210556af70f\") " pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" Jul 24 15:47:29.104973 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:29.104838 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/973ac0cb-cb79-47fd-af8b-b210556af70f-podres\") pod \"perf-node-gather-daemonset-g4z5l\" (UID: \"973ac0cb-cb79-47fd-af8b-b210556af70f\") " pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" Jul 24 15:47:29.104973 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:29.104902 2531 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4gmn\" (UniqueName: \"kubernetes.io/projected/973ac0cb-cb79-47fd-af8b-b210556af70f-kube-api-access-j4gmn\") pod \"perf-node-gather-daemonset-g4z5l\" (UID: \"973ac0cb-cb79-47fd-af8b-b210556af70f\") " pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" Jul 24 15:47:29.105072 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:29.104975 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/973ac0cb-cb79-47fd-af8b-b210556af70f-podres\") pod \"perf-node-gather-daemonset-g4z5l\" (UID: \"973ac0cb-cb79-47fd-af8b-b210556af70f\") " pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" Jul 24 15:47:29.115345 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:29.115318 2531 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4gmn\" (UniqueName: \"kubernetes.io/projected/973ac0cb-cb79-47fd-af8b-b210556af70f-kube-api-access-j4gmn\") pod \"perf-node-gather-daemonset-g4z5l\" (UID: \"973ac0cb-cb79-47fd-af8b-b210556af70f\") " pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" Jul 24 15:47:29.220838 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:29.220798 2531 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" Jul 24 15:47:29.383129 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:29.383089 2531 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l"] Jul 24 15:47:29.384472 ip-10-0-137-108 kubenswrapper[2531]: W0724 15:47:29.384446 2531 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod973ac0cb_cb79_47fd_af8b_b210556af70f.slice/crio-09929d444ee5b43cba1a96cc4d11ea9372d309e2474584a6234a1875b058b9a7 WatchSource:0}: Error finding container 09929d444ee5b43cba1a96cc4d11ea9372d309e2474584a6234a1875b058b9a7: Status 404 returned error can't find the container with id 09929d444ee5b43cba1a96cc4d11ea9372d309e2474584a6234a1875b058b9a7 Jul 24 15:47:30.088456 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:30.088418 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" event={"ID":"973ac0cb-cb79-47fd-af8b-b210556af70f","Type":"ContainerStarted","Data":"ed453e8c0cfa31d4fff9fa63875ca8a33f520e0df7c09ef558f1307cdc17a88d"} Jul 24 15:47:30.088654 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:30.088488 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" Jul 24 15:47:30.088654 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:30.088506 2531 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" event={"ID":"973ac0cb-cb79-47fd-af8b-b210556af70f","Type":"ContainerStarted","Data":"09929d444ee5b43cba1a96cc4d11ea9372d309e2474584a6234a1875b058b9a7"} Jul 24 15:47:30.114631 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:30.114563 2531 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" podStartSLOduration=2.114547772 podStartE2EDuration="2.114547772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:47:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:47:30.112893696 +0000 UTC m=+3082.343370378" watchObservedRunningTime="2026-07-24 15:47:30.114547772 +0000 UTC m=+3082.345024450" Jul 24 15:47:30.439158 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:30.439122 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-dntl4_020f32d2-c323-430c-b6a6-d97a25182801/networking-console-plugin/0.log" Jul 24 15:47:31.164430 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:31.164399 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/1.log" Jul 24 15:47:31.174917 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:31.170005 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-77k6k_520066e2-8b42-4872-b257-20df38f77e37/console-operator/2.log" Jul 24 15:47:31.676298 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:31.676267 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-dd46c778-5xtfc_62d656ea-2f68-4fb8-8b9b-8bde992ae7d4/console/0.log" Jul 24 15:47:32.379681 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:32.379654 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-ndjm9_72091411-787f-478a-a97f-8ebc53464929/volume-data-source-validator/0.log" Jul 24 15:47:33.483005 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:33.482977 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2jdtw_31f123ed-b090-4d80-a46b-888032edc051/dns/0.log" Jul 24 15:47:33.512080 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:33.512049 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2jdtw_31f123ed-b090-4d80-a46b-888032edc051/kube-rbac-proxy/0.log" Jul 24 15:47:33.671037 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:33.671008 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hcxz7_bdaadab4-2a42-46d7-8ca6-24881a9d54d0/dns-node-resolver/0.log" Jul 24 15:47:34.413044 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:34.413010 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pbnwh_91255a26-e33b-408d-81d1-b8ec24d1e7d6/node-ca/0.log" Jul 24 15:47:35.763644 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:35.763615 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-758f749c7-9xg27_f32962bc-3bd3-41f4-89fe-245a37e1582f/kube-auth-proxy/0.log" Jul 24 15:47:35.800289 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:35.800257 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-gzdst_0e873231-3e07-4d1d-9d0d-bf7cb190d9c3/istio-proxy/0.log" Jul 24 15:47:35.829716 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:35.829659 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-gzdst_0e873231-3e07-4d1d-9d0d-bf7cb190d9c3/istio-proxy/1.log" Jul 24 15:47:35.903153 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:35.903121 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-hfcxx_58138eae-76c2-489f-abdb-f858b5ef0108/payload-processing/0.log" Jul 24 15:47:35.928468 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:35.928438 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-776fc686df-t8g55_3ede17c9-5194-41b7-a4e6-55a258fe1a4e/router/0.log" Jul 24 15:47:36.107571 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:36.107497 2531 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fd7vt/perf-node-gather-daemonset-g4z5l" Jul 24 15:47:36.846122 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:36.846088 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s4b7h_9e5d1043-9361-463b-a342-bcb81a9f181d/serve-healthcheck-canary/0.log" Jul 24 15:47:37.442970 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:37.442934 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-wjxvg_595851c1-64b8-44ad-b819-f6b635101e2e/insights-operator/0.log" Jul 24 15:47:37.444035 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:37.443973 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-wjxvg_595851c1-64b8-44ad-b819-f6b635101e2e/insights-operator/1.log" Jul 24 15:47:37.695765 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:37.695681 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p9x5t_54da61a7-9d56-41ec-8536-b1e4bfe30fb3/kube-rbac-proxy/0.log" Jul 24 15:47:37.747964 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:37.747933 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p9x5t_54da61a7-9d56-41ec-8536-b1e4bfe30fb3/exporter/0.log" Jul 24 15:47:37.793013 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:37.792981 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p9x5t_54da61a7-9d56-41ec-8536-b1e4bfe30fb3/extractor/0.log" Jul 24 15:47:39.958984 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:39.958954 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-758f749c7-9xg27_f32962bc-3bd3-41f4-89fe-245a37e1582f/kube-auth-proxy/0.log" Jul 24 15:47:39.985788 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:39.985742 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-gzdst_0e873231-3e07-4d1d-9d0d-bf7cb190d9c3/istio-proxy/0.log" Jul 24 15:47:40.013237 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:40.013174 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-gzdst_0e873231-3e07-4d1d-9d0d-bf7cb190d9c3/istio-proxy/1.log" Jul 24 15:47:40.079567 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:40.079533 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-hfcxx_58138eae-76c2-489f-abdb-f858b5ef0108/payload-processing/0.log" Jul 24 15:47:40.097554 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:40.097527 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-776fc686df-t8g55_3ede17c9-5194-41b7-a4e6-55a258fe1a4e/router/0.log" Jul 24 15:47:40.359995 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:40.359962 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-ng4q9_1d69f3f6-a0cc-45aa-8d2d-cc0b4cf5017b/manager/0.log" Jul 24 15:47:40.575639 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:40.575606 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-cs6dm_6060f538-1807-499a-b224-e07dffbebbe8/manager/0.log" Jul 24 15:47:40.837289 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:40.837196 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-zk72b_541aed6a-215c-465a-b8ff-dfcd57c9f107/server/0.log" Jul 24 15:47:40.964537 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:40.964507 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f5584866b-6mp2c_077904ac-fa0f-4a6c-b19e-9e9b6da1b68f/manager/0.log" Jul 24 15:47:40.989913 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:40.989861 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f5584866b-6mp2c_077904ac-fa0f-4a6c-b19e-9e9b6da1b68f/copy-manifests/0.log" Jul 24 15:47:41.675436 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:41.675406 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-758f749c7-9xg27_f32962bc-3bd3-41f4-89fe-245a37e1582f/kube-auth-proxy/0.log" Jul 24 15:47:41.712599 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:41.712566 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-gzdst_0e873231-3e07-4d1d-9d0d-bf7cb190d9c3/istio-proxy/0.log" Jul 24 15:47:41.738705 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:41.738672 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-gzdst_0e873231-3e07-4d1d-9d0d-bf7cb190d9c3/istio-proxy/1.log" Jul 24 15:47:41.788632 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:41.788603 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-hfcxx_58138eae-76c2-489f-abdb-f858b5ef0108/payload-processing/0.log" Jul 24 15:47:41.805491 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:41.805460 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-776fc686df-t8g55_3ede17c9-5194-41b7-a4e6-55a258fe1a4e/router/0.log" Jul 24 15:47:42.526255 ip-10-0-137-108 kubenswrapper[2531]: I0724 15:47:42.526223 2531 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-gdm44_94b58c1f-e723-4430-8847-0891446cb82d/openshift-lws-operator/0.log"