Jul 08 21:30:48.534404 ip-10-0-139-229 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 21:30:48.534420 ip-10-0-139-229 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 21:30:48.534430 ip-10-0-139-229 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 21:30:48.534776 ip-10-0-139-229 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 21:30:58.696928 ip-10-0-139-229 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 21:30:58.696944 ip-10-0-139-229 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot cf2a6c26c50e4f4b8401c3048655707a -- Jul 08 21:32:56.530440 ip-10-0-139-229 systemd[1]: Starting Kubernetes Kubelet... Jul 08 21:32:56.978149 ip-10-0-139-229 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 21:32:56.978149 ip-10-0-139-229 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 21:32:56.978149 ip-10-0-139-229 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 21:32:56.978149 ip-10-0-139-229 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 21:32:56.978149 ip-10-0-139-229 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 21:32:56.978805 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.978167 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 21:32:56.983998 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.983975 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:32:56.983998 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.983994 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:32:56.983998 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.983998 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:32:56.983998 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984002 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:32:56.983998 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984005 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:32:56.984217 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984008 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:32:56.984217 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984011 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:32:56.984217 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984014 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:32:56.984217 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984017 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:32:56.984217 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984019 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:32:56.984217 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984022 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:32:56.984217 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984025 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:32:56.984217 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984028 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:32:56.984217 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984031 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:32:56.984217 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984033 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:32:56.984217 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984036 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:32:56.984217 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984039 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:32:56.984217 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984041 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:32:56.984217 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984044 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:32:56.984217 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984047 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:32:56.984217 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984049 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:32:56.984217 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984051 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:32:56.984217 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984054 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:32:56.984217 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984057 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:32:56.984217 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984060 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:32:56.984696 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984062 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:32:56.984696 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984065 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:32:56.984696 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984067 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:32:56.984696 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984070 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:32:56.984696 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984072 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:32:56.984696 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984075 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:32:56.984696 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984078 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:32:56.984696 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984083 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:32:56.984696 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984086 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:32:56.984696 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984089 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:32:56.984696 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984091 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:32:56.984696 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984094 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:32:56.984696 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984096 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:32:56.984696 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984100 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:32:56.984696 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984102 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:32:56.984696 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984105 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:32:56.984696 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984108 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:32:56.984696 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984110 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:32:56.984696 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984113 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:32:56.984696 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984116 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:32:56.985168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984118 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:32:56.985168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984120 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:32:56.985168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984123 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:32:56.985168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984126 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:32:56.985168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984128 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:32:56.985168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984131 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:32:56.985168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984133 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:32:56.985168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984136 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:32:56.985168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984141 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:32:56.985168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984144 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:32:56.985168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984148 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:32:56.985168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984153 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:32:56.985168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984156 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:32:56.985168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984160 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:32:56.985168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984162 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:32:56.985168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984165 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:32:56.985168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984170 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:32:56.985168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984173 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:32:56.985615 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984192 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:32:56.985615 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984196 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:32:56.985615 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984198 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:32:56.985615 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984202 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:32:56.985615 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984205 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:32:56.985615 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984208 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:32:56.985615 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984211 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:32:56.985615 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984216 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:32:56.985615 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984219 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:32:56.985615 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984222 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:32:56.985615 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984225 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:32:56.985615 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984228 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:32:56.985615 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984231 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:32:56.985615 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984235 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:32:56.985615 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984238 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:32:56.985615 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984241 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:32:56.985615 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984244 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:32:56.985615 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984247 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:32:56.985615 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984250 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:32:56.985615 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984253 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:32:56.986137 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984256 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:32:56.986137 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984259 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:32:56.986137 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984262 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:32:56.986137 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984265 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:32:56.986137 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984267 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:32:56.986137 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984270 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:32:56.986137 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984273 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:32:56.986137 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984276 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:32:56.986137 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984278 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:32:56.986137 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984281 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:32:56.986137 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984283 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:32:56.986137 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984286 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:32:56.986137 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984288 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:32:56.986137 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984291 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:32:56.986137 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984293 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:32:56.986137 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.984296 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:32:56.986137 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985019 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:32:56.986137 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985027 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:32:56.986137 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985031 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:32:56.986137 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985037 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:32:56.986626 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985040 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:32:56.986626 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985043 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:32:56.986626 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985046 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:32:56.986626 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985049 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:32:56.986626 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985052 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:32:56.986626 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985055 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:32:56.986626 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985058 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:32:56.986626 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985061 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:32:56.986626 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985063 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:32:56.986626 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985066 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:32:56.986626 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985068 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:32:56.986626 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985071 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:32:56.986626 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985073 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:32:56.986626 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985077 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:32:56.986626 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985079 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:32:56.986626 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985082 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:32:56.986626 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985086 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:32:56.986626 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985090 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:32:56.986626 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985092 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:32:56.987096 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985095 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:32:56.987096 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985098 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:32:56.987096 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985101 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:32:56.987096 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985104 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:32:56.987096 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985106 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:32:56.987096 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985109 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:32:56.987096 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985111 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:32:56.987096 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985114 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:32:56.987096 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985116 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:32:56.987096 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985119 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:32:56.987096 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985122 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:32:56.987096 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985126 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:32:56.987096 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985130 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:32:56.987096 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985132 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:32:56.987096 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985135 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:32:56.987096 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985138 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:32:56.987096 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985140 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:32:56.987096 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985143 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:32:56.987096 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985145 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:32:56.987096 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985148 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:32:56.987596 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985150 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:32:56.987596 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985153 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:32:56.987596 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985156 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:32:56.987596 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985159 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:32:56.987596 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985163 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:32:56.987596 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985165 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:32:56.987596 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985168 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:32:56.987596 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985172 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:32:56.987596 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985174 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:32:56.987596 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985193 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:32:56.987596 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985196 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:32:56.987596 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985199 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:32:56.987596 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985201 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:32:56.987596 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985204 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:32:56.987596 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985206 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:32:56.987596 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985209 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:32:56.987596 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985212 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:32:56.987596 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985214 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:32:56.987596 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985217 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985219 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985222 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985225 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985227 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985230 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985234 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985237 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985240 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985242 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985245 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985247 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985250 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985253 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985256 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985260 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985262 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985265 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985268 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985270 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985273 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:32:56.988060 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985275 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:32:56.988676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985278 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:32:56.988676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985281 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:32:56.988676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985284 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:32:56.988676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985286 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:32:56.988676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985289 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:32:56.988676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985291 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:32:56.988676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985294 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:32:56.988676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985297 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:32:56.988676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985301 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:32:56.988676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985304 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:32:56.988676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985307 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:32:56.988676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985310 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:32:56.988676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985313 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:32:56.988676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985316 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:32:56.988676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985319 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:32:56.988676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985322 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:32:56.988676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985686 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:32:56.988676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985691 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:32:56.988676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985694 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:32:56.989168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985696 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:32:56.989168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985699 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:32:56.989168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985701 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:32:56.989168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985704 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:32:56.989168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985707 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:32:56.989168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985710 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:32:56.989168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985713 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:32:56.989168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985716 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:32:56.989168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985718 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:32:56.989168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985721 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:32:56.989168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985723 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:32:56.989168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985727 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:32:56.989168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985731 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:32:56.989168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985734 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:32:56.989168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985737 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:32:56.989168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985740 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:32:56.989168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985743 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:32:56.989168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985746 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:32:56.989168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985748 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:32:56.989168 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985751 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:32:56.989666 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985754 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:32:56.989666 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985757 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:32:56.989666 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985759 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:32:56.989666 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985762 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:32:56.989666 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985764 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:32:56.989666 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985767 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:32:56.989666 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985769 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:32:56.989666 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985772 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:32:56.989666 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985774 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:32:56.989666 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985779 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:32:56.989666 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985782 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:32:56.989666 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985784 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:32:56.989666 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985787 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:32:56.989666 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985789 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:32:56.989666 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985792 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:32:56.989666 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985795 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:32:56.989666 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985797 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:32:56.989666 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985800 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:32:56.989666 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985803 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:32:56.990135 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985805 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:32:56.990135 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985808 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:32:56.990135 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985810 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:32:56.990135 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985813 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:32:56.990135 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985816 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:32:56.990135 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985819 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:32:56.990135 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985821 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:32:56.990135 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985824 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:32:56.990135 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985827 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:32:56.990135 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985829 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:32:56.990135 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985832 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:32:56.990135 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985834 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:32:56.990135 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985838 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:32:56.990135 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985841 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:32:56.990135 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985844 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:32:56.990135 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985846 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:32:56.990135 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985849 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:32:56.990135 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985852 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:32:56.990135 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985855 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:32:56.990135 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985858 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:32:56.990612 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985863 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:32:56.990612 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985867 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:32:56.990612 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985871 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:32:56.990612 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985873 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:32:56.990612 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985876 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:32:56.990612 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985879 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:32:56.990612 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985881 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:32:56.990612 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985884 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:32:56.990612 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985887 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:32:56.990612 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985889 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:32:56.990612 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985892 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:32:56.990612 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985894 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:32:56.990612 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985897 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:32:56.990612 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985900 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:32:56.990612 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985902 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:32:56.990612 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985904 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:32:56.990612 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985907 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:32:56.990612 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985910 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:32:56.990612 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985912 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:32:56.990612 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985915 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985917 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985920 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985923 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985925 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985928 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985930 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985933 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985935 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985938 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985941 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985944 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985946 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985949 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985951 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985956 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985958 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.985961 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986029 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986036 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986042 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 21:32:56.991114 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986046 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986050 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986054 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986058 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986061 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986065 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986067 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986071 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986074 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986077 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986080 2580 flags.go:64] FLAG: --cgroup-root="" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986083 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986086 2580 flags.go:64] FLAG: --client-ca-file="" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986089 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986092 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986097 2580 flags.go:64] FLAG: --cluster-domain="" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986100 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986103 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986106 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986109 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986112 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986118 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986121 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 21:32:56.991629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986124 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986127 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986130 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986134 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986138 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986141 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986145 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986148 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986151 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986154 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986157 2580 flags.go:64] FLAG: --enable-server="true" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986160 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986164 2580 flags.go:64] FLAG: --event-burst="100" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986167 2580 flags.go:64] FLAG: --event-qps="50" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986169 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986172 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986175 2580 flags.go:64] FLAG: --eviction-hard="" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986193 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986196 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986199 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986202 2580 flags.go:64] FLAG: --eviction-soft="" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986205 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986208 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986212 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986214 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 21:32:56.992160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986217 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986220 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986223 2580 flags.go:64] FLAG: --feature-gates="" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986226 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986229 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986233 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986236 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986239 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986242 2580 flags.go:64] FLAG: --help="false" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986245 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-139-229.ec2.internal" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986249 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986252 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986254 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986258 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986261 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986263 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986266 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986269 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986272 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986275 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986278 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986280 2580 flags.go:64] FLAG: --kube-reserved="" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986283 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986286 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 21:32:56.992824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986289 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986291 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986294 2580 flags.go:64] FLAG: --lock-file="" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986297 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986299 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986302 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986307 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986310 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986313 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986316 2580 flags.go:64] FLAG: --logging-format="text" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986319 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986322 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986325 2580 flags.go:64] FLAG: --manifest-url="" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986329 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986335 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986338 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986342 2580 flags.go:64] FLAG: --max-pods="110" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986345 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986348 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986351 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986354 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986357 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986360 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986363 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986370 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 21:32:56.993475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986373 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986376 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986379 2580 flags.go:64] FLAG: --pod-cidr="" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986382 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986387 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986390 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986393 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986396 2580 flags.go:64] FLAG: --port="10250" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986399 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986402 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-020f2cde96dd1d581" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986405 2580 flags.go:64] FLAG: --qos-reserved="" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986408 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986411 2580 flags.go:64] FLAG: --register-node="true" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986414 2580 flags.go:64] FLAG: --register-with-taints="" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986417 2580 flags.go:64] FLAG: --registry-burst="10" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986420 2580 flags.go:64] FLAG: --registry-qps="5" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986423 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986426 2580 flags.go:64] FLAG: --reserved-memory="" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986429 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986432 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986435 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986439 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986442 2580 flags.go:64] FLAG: --runonce="false" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986445 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986448 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 21:32:56.994076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986451 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986454 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986456 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986459 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986462 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986465 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986468 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986470 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986473 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986476 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986479 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986482 2580 flags.go:64] FLAG: --system-cgroups="" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986484 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986489 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986492 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.986495 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.987194 2580 flags.go:64] FLAG: --tls-min-version="" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.987198 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.987201 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.987204 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.987207 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.987210 2580 flags.go:64] FLAG: --v="2" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.987214 2580 flags.go:64] FLAG: --version="false" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.987219 2580 flags.go:64] FLAG: --vmodule="" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.987222 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 21:32:56.994702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.987225 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 21:32:56.995336 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987355 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:32:56.995336 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987360 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:32:56.995336 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987364 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:32:56.995336 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987366 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:32:56.995336 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987369 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:32:56.995336 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987372 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:32:56.995336 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987375 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:32:56.995336 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987378 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:32:56.995336 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987381 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:32:56.995336 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987384 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:32:56.995336 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987386 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:32:56.995336 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987389 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:32:56.995336 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987391 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:32:56.995336 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987394 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:32:56.995336 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987396 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:32:56.995336 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987399 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:32:56.995336 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987402 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:32:56.995336 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987405 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:32:56.995336 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987407 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:32:56.995336 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987410 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:32:56.995821 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987412 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:32:56.995821 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987415 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:32:56.995821 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987419 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:32:56.995821 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987422 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:32:56.995821 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987425 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:32:56.995821 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987428 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:32:56.995821 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987430 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:32:56.995821 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987433 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:32:56.995821 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987435 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:32:56.995821 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987438 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:32:56.995821 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987441 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:32:56.995821 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987443 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:32:56.995821 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987446 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:32:56.995821 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987449 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:32:56.995821 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987451 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:32:56.995821 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987454 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:32:56.995821 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987456 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:32:56.995821 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987459 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:32:56.995821 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987463 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:32:56.996287 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987466 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:32:56.996287 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987468 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:32:56.996287 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987471 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:32:56.996287 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987473 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:32:56.996287 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987476 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:32:56.996287 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987478 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:32:56.996287 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987481 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:32:56.996287 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987483 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:32:56.996287 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987486 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:32:56.996287 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987488 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:32:56.996287 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987491 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:32:56.996287 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987493 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:32:56.996287 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987496 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:32:56.996287 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987498 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:32:56.996287 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987502 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:32:56.996287 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987506 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:32:56.996287 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987509 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:32:56.996287 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987512 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:32:56.996287 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987515 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:32:56.996287 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987517 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:32:56.996769 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987521 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:32:56.996769 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987524 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:32:56.996769 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987527 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:32:56.996769 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987530 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:32:56.996769 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987534 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:32:56.996769 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987542 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:32:56.996769 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987545 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:32:56.996769 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987547 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:32:56.996769 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987550 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:32:56.996769 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987552 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:32:56.996769 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987555 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:32:56.996769 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987560 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:32:56.996769 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987562 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:32:56.996769 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987565 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:32:56.996769 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987567 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:32:56.996769 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987570 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:32:56.996769 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987573 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:32:56.996769 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987576 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:32:56.996769 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987578 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:32:56.997247 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987581 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:32:56.997247 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987583 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:32:56.997247 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987586 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:32:56.997247 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987588 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:32:56.997247 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987591 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:32:56.997247 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987593 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:32:56.997247 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987596 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:32:56.997247 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987599 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:32:56.997247 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987601 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:32:56.997247 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987604 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:32:56.997247 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987607 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:32:56.997247 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987609 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:32:56.997247 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987612 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:32:56.997247 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987614 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:32:56.997247 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987617 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:32:56.997247 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987620 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:32:56.997247 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987622 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:32:56.997247 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987625 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:32:56.997247 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987628 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:32:56.997247 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987631 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:32:56.997767 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.987634 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:32:56.997767 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.987643 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 21:32:56.997767 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.994625 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 21:32:56.997767 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.994638 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 21:32:56.997767 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.994651 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 21:32:56.997767 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.994658 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 21:32:56.997767 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994717 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:32:56.997767 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994723 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:32:56.997767 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994728 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:32:56.997767 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994731 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:32:56.997767 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994734 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:32:56.997767 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994737 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:32:56.997767 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994740 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:32:56.997767 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994743 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:32:56.997767 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994746 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:32:56.998132 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994749 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:32:56.998132 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994752 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:32:56.998132 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994754 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:32:56.998132 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994757 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:32:56.998132 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994759 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:32:56.998132 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994762 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:32:56.998132 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994774 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:32:56.998132 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994777 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:32:56.998132 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994779 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:32:56.998132 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994782 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:32:56.998132 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994785 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:32:56.998132 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994787 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:32:56.998132 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994790 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:32:56.998132 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994792 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:32:56.998132 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994795 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:32:56.998132 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994797 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:32:56.998132 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994800 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:32:56.998132 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994803 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:32:56.998132 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994805 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:32:56.998132 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994808 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:32:56.998676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994818 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:32:56.998676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994821 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:32:56.998676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994824 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:32:56.998676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994826 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:32:56.998676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994829 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:32:56.998676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994831 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:32:56.998676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994834 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:32:56.998676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994836 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:32:56.998676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994839 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:32:56.998676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994841 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:32:56.998676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994844 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:32:56.998676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994846 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:32:56.998676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994849 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:32:56.998676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994851 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:32:56.998676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994854 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:32:56.998676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994856 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:32:56.998676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994859 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:32:56.998676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994861 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:32:56.998676 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994864 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:32:56.999146 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994866 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:32:56.999146 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994868 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:32:56.999146 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994871 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:32:56.999146 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994873 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:32:56.999146 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994876 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:32:56.999146 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994878 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:32:56.999146 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994881 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:32:56.999146 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994883 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:32:56.999146 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994886 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:32:56.999146 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994888 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:32:56.999146 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994891 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:32:56.999146 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994894 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:32:56.999146 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994898 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:32:56.999146 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994901 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:32:56.999146 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994910 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:32:56.999146 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994913 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:32:56.999146 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994915 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:32:56.999146 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994918 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:32:56.999146 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994920 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:32:56.999622 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994924 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:32:56.999622 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994927 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:32:56.999622 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994930 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:32:56.999622 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994933 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:32:56.999622 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994936 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:32:56.999622 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994939 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:32:56.999622 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994942 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:32:56.999622 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994945 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:32:56.999622 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994947 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:32:56.999622 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994950 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:32:56.999622 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994953 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:32:56.999622 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994955 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:32:56.999622 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994958 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:32:56.999622 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994960 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:32:56.999622 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994963 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:32:56.999622 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994965 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:32:56.999622 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994968 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:32:56.999622 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994970 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:32:56.999622 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994973 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:32:57.000089 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994975 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:32:57.000089 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994977 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:32:57.000089 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994980 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:32:57.000089 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994982 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:32:57.000089 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994985 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:32:57.000089 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994987 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:32:57.000089 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994990 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:32:57.000089 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994993 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:32:57.000089 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.994995 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:32:57.000089 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995003 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:32:57.000089 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995006 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:32:57.000089 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995009 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:32:57.000089 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995011 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:32:57.000089 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.995016 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 21:32:57.000089 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995149 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 21:32:57.000466 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995154 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 21:32:57.000466 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995157 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 21:32:57.000466 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995160 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 21:32:57.000466 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995165 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 21:32:57.000466 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995168 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 21:32:57.000466 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995170 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 21:32:57.000466 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995173 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 21:32:57.000466 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995192 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 21:32:57.000466 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995196 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 21:32:57.000466 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995199 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 21:32:57.000466 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995202 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 21:32:57.000466 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995205 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 21:32:57.000466 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995207 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 21:32:57.000466 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995211 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 21:32:57.000466 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995213 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 21:32:57.000466 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995222 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 21:32:57.000466 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995225 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 21:32:57.000466 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995228 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 21:32:57.000466 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995230 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 21:32:57.000917 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995233 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 21:32:57.000917 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995236 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 21:32:57.000917 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995238 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 21:32:57.000917 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995241 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 21:32:57.000917 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995244 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 21:32:57.000917 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995246 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 21:32:57.000917 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995250 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 21:32:57.000917 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995254 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 21:32:57.000917 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995262 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 21:32:57.000917 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995265 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 08 21:32:57.000917 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995268 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 21:32:57.000917 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995271 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 21:32:57.000917 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995274 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 21:32:57.000917 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995276 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 21:32:57.000917 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995279 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 21:32:57.000917 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995282 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 21:32:57.000917 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995285 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 21:32:57.000917 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995287 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 21:32:57.000917 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995290 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 21:32:57.001417 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995293 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 21:32:57.001417 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995297 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 21:32:57.001417 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995299 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 21:32:57.001417 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995302 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 21:32:57.001417 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995305 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 21:32:57.001417 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995307 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 21:32:57.001417 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995310 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 21:32:57.001417 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995312 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 21:32:57.001417 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995315 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 21:32:57.001417 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995317 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 21:32:57.001417 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995320 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 21:32:57.001417 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995322 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 21:32:57.001417 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995325 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 21:32:57.001417 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995327 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 21:32:57.001417 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995330 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 21:32:57.001417 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995333 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 21:32:57.001417 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995335 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 21:32:57.001417 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995338 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 21:32:57.001417 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995341 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 21:32:57.001869 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995343 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 21:32:57.001869 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995346 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 21:32:57.001869 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995349 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 21:32:57.001869 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995357 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 21:32:57.001869 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995360 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 21:32:57.001869 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995362 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 21:32:57.001869 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995364 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 21:32:57.001869 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995367 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 21:32:57.001869 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995369 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 21:32:57.001869 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995372 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 21:32:57.001869 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995374 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 21:32:57.001869 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995376 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 21:32:57.001869 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995379 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 21:32:57.001869 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995381 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 21:32:57.001869 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995384 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 21:32:57.001869 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995386 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 21:32:57.001869 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995388 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 21:32:57.001869 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995391 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 21:32:57.001869 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995393 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 21:32:57.001869 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995395 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 21:32:57.002367 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995398 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 21:32:57.002367 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995400 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 21:32:57.002367 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995403 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 21:32:57.002367 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995405 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 21:32:57.002367 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995408 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 21:32:57.002367 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995410 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 21:32:57.002367 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995412 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 21:32:57.002367 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995417 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 21:32:57.002367 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995419 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 21:32:57.002367 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995422 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 21:32:57.002367 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995425 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 21:32:57.002367 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995428 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 21:32:57.002367 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995430 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 21:32:57.002367 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995432 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 21:32:57.002367 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995435 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 21:32:57.002367 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995437 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 21:32:57.002367 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995445 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 21:32:57.002367 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995448 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 21:32:57.002367 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995451 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 21:32:57.002367 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995453 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 21:32:57.002884 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:56.995456 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 21:32:57.002884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.995461 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 21:32:57.002884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.995573 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 21:32:57.002884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.998250 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 21:32:57.002884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.999830 2580 server.go:1019] "Starting client certificate rotation" Jul 08 21:32:57.002884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.999936 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 21:32:57.002884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:56.999968 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 21:32:57.002884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.002464 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 21:32:57.005970 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.005950 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 21:32:57.015497 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.015479 2580 log.go:25] "Validated CRI v1 runtime API" Jul 08 21:32:57.017551 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.017536 2580 log.go:25] "Validated CRI v1 image API" Jul 08 21:32:57.018413 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.018398 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 21:32:57.023894 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.023875 2580 fs.go:135] Filesystem UUIDs: map[41c1b63f-75bc-41d7-9a3a-95c6dc3d1e44:/dev/nvme0n1p3 54d4b4b5-a762-42f2-a13c-d465b8681e02:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 08 21:32:57.023969 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.023894 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 21:32:57.028337 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.028228 2580 manager.go:216] Machine: {Timestamp:2026-07-08 21:32:57.027664252 +0000 UTC m=+0.374179818 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098847 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f6f731856a2cba43dc6ff6de27997 SystemUUID:ec2f6f73-1856-a2cb-a43d-c6ff6de27997 BootID:cf2a6c26-c50e-4f4b-8401-c3048655707a Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:94:dd:b2:9d:a3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:94:dd:b2:9d:a3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:0f:85:7b:83:bd Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 21:32:57.028337 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.028333 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 21:32:57.028443 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.028431 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 21:32:57.031673 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.031645 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 21:32:57.031811 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.031678 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-229.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 21:32:57.032342 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.032332 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 21:32:57.032371 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.032344 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 21:32:57.032371 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.032352 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 21:32:57.032371 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.032363 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 21:32:57.033849 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.033838 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 21:32:57.037141 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.037129 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 08 21:32:57.037266 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.037257 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 21:32:57.037308 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.037289 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 08 21:32:57.037340 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.037315 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 21:32:57.037340 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.037332 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 21:32:57.037410 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.037344 2580 kubelet.go:398] "Adding apiserver pod source" Jul 08 21:32:57.037410 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.037357 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 21:32:57.037495 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.037404 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 21:32:57.037849 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.037827 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 21:32:57.037910 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.037853 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 21:32:57.040791 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.040772 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 21:32:57.042641 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.042616 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 21:32:57.042690 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.042659 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 21:32:57.043805 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.043794 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 21:32:57.043837 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.043810 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 21:32:57.043837 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.043816 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 21:32:57.043837 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.043823 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 21:32:57.043837 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.043836 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 21:32:57.043987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.043844 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 21:32:57.043987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.043853 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 21:32:57.043987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.043858 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 21:32:57.043987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.043864 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 21:32:57.043987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.043869 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 21:32:57.043987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.043885 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 21:32:57.043987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.043890 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 21:32:57.044653 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.044643 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 21:32:57.044653 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.044654 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 21:32:57.050154 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.050118 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 21:32:57.050258 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.050155 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-229.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 21:32:57.050258 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.050204 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-229.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 21:32:57.050370 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.050269 2580 server.go:1268] "Started kubelet" Jul 08 21:32:57.050370 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.050308 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 21:32:57.050457 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.050392 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 21:32:57.050500 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.050460 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 21:32:57.051155 ip-10-0-139-229 systemd[1]: Started Kubernetes Kubelet. Jul 08 21:32:57.051639 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.051621 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 08 21:32:57.052123 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.052102 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 21:32:57.057460 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.057430 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 21:32:57.058046 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.058035 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 21:32:57.058114 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.058089 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 21:32:57.058573 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.058295 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-229.ec2.internal\" not found" Jul 08 21:32:57.058694 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.058677 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 21:32:57.058777 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.058716 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 21:32:57.058777 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.058728 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 21:32:57.058884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.058844 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 08 21:32:57.058884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.058856 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 08 21:32:57.058956 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.058923 2580 factory.go:55] Registering systemd factory Jul 08 21:32:57.058956 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.058934 2580 factory.go:223] Registration of the systemd container factory successfully Jul 08 21:32:57.059161 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.059151 2580 factory.go:153] Registering CRI-O factory Jul 08 21:32:57.059221 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.059163 2580 factory.go:223] Registration of the crio container factory successfully Jul 08 21:32:57.059262 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.059239 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 21:32:57.059299 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.059279 2580 factory.go:103] Registering Raw factory Jul 08 21:32:57.059299 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.059296 2580 manager.go:1196] Started watching for new ooms in manager Jul 08 21:32:57.059495 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.058230 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-229.ec2.internal.18c06dfaf4839200 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-229.ec2.internal,UID:ip-10-0-139-229.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-229.ec2.internal,},FirstTimestamp:2026-07-08 21:32:57.050231296 +0000 UTC m=+0.396746866,LastTimestamp:2026-07-08 21:32:57.050231296 +0000 UTC m=+0.396746866,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-229.ec2.internal,}" Jul 08 21:32:57.059798 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.059782 2580 manager.go:349] Starting recovery of all containers Jul 08 21:32:57.069638 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.069435 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 21:32:57.069813 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.069762 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-229.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 21:32:57.070773 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.070744 2580 manager.go:354] Recovery completed Jul 08 21:32:57.077072 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.077056 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:32:57.078830 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.078804 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:32:57.078907 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.078847 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:32:57.078907 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.078861 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:32:57.079460 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.079446 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gc888" Jul 08 21:32:57.080245 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.080229 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 21:32:57.080245 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.080239 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:32:57.080403 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.080242 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 21:32:57.080403 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.080268 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 08 21:32:57.080915 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.080901 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:32:57.080967 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.080926 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:32:57.080967 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.080936 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:32:57.081513 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.081501 2580 policy_none.go:49] "None policy: Start" Jul 08 21:32:57.081567 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.081516 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 21:32:57.081567 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.081526 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 21:32:57.082171 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.082103 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-229.ec2.internal.18c06dfaf637f379 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-229.ec2.internal,UID:ip-10-0-139-229.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-229.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-229.ec2.internal,},FirstTimestamp:2026-07-08 21:32:57.078829945 +0000 UTC m=+0.425345511,LastTimestamp:2026-07-08 21:32:57.078829945 +0000 UTC m=+0.425345511,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-229.ec2.internal,}" Jul 08 21:32:57.082582 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.082569 2580 policy_none.go:47] "Start" Jul 08 21:32:57.098243 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.098221 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gc888" Jul 08 21:32:57.099740 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.099680 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-229.ec2.internal.18c06dfaf6384f9e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-229.ec2.internal,UID:ip-10-0-139-229.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-229.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-229.ec2.internal,},FirstTimestamp:2026-07-08 21:32:57.078853534 +0000 UTC m=+0.425369101,LastTimestamp:2026-07-08 21:32:57.078853534 +0000 UTC m=+0.425369101,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-229.ec2.internal,}" Jul 08 21:32:57.120728 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.120715 2580 manager.go:337] "Starting Device Plugin manager" Jul 08 21:32:57.120813 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.120743 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 21:32:57.120813 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.120751 2580 server.go:87] "Starting device plugin registration server" Jul 08 21:32:57.120990 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.120977 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 21:32:57.121027 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.120994 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 21:32:57.121162 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.121141 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 21:32:57.121312 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.121288 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 21:32:57.121312 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.121304 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 21:32:57.121685 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.121670 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 21:32:57.121748 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.121723 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-229.ec2.internal\" not found" Jul 08 21:32:57.186250 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.186220 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 21:32:57.188337 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.187514 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 21:32:57.188337 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.187545 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 21:32:57.188337 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.187566 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 21:32:57.188337 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.187597 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 21:32:57.192334 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.192312 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:32:57.221375 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.221344 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:32:57.222377 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.222362 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:32:57.222458 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.222395 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:32:57.222458 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.222410 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:32:57.222458 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.222436 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-229.ec2.internal" Jul 08 21:32:57.235056 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.235012 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-229.ec2.internal" Jul 08 21:32:57.235056 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.235033 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-229.ec2.internal\": node \"ip-10-0-139-229.ec2.internal\" not found" Jul 08 21:32:57.286074 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.286051 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-229.ec2.internal\" not found" Jul 08 21:32:57.288139 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.288123 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-229.ec2.internal"] Jul 08 21:32:57.288203 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.288193 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:32:57.289654 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.289640 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:32:57.289702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.289669 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:32:57.289702 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.289681 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:32:57.291901 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.291889 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:32:57.292017 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.292002 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal" Jul 08 21:32:57.292054 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.292034 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:32:57.292628 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.292610 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:32:57.292698 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.292640 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:32:57.292698 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.292649 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:32:57.292698 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.292614 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:32:57.292790 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.292718 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:32:57.292790 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.292730 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:32:57.294848 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.294830 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-229.ec2.internal" Jul 08 21:32:57.294848 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.294853 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 21:32:57.295494 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.295477 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasSufficientMemory" Jul 08 21:32:57.295597 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.295506 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 21:32:57.295597 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.295522 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeHasSufficientPID" Jul 08 21:32:57.315690 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.315669 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-229.ec2.internal\" not found" node="ip-10-0-139-229.ec2.internal" Jul 08 21:32:57.319922 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.319905 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-229.ec2.internal\" not found" node="ip-10-0-139-229.ec2.internal" Jul 08 21:32:57.386917 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.386894 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-229.ec2.internal\" not found" Jul 08 21:32:57.459661 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.459608 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b2bbc34fbe55f79f66e34f21916774a0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal\" (UID: \"b2bbc34fbe55f79f66e34f21916774a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal" Jul 08 21:32:57.459774 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.459663 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2bbc34fbe55f79f66e34f21916774a0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal\" (UID: \"b2bbc34fbe55f79f66e34f21916774a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal" Jul 08 21:32:57.459774 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.459702 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4a1b4940af681a5e34bbf5436c61df86-config\") pod \"kube-apiserver-proxy-ip-10-0-139-229.ec2.internal\" (UID: \"4a1b4940af681a5e34bbf5436c61df86\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-229.ec2.internal" Jul 08 21:32:57.487726 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.487676 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-229.ec2.internal\" not found" Jul 08 21:32:57.560393 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.560368 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b2bbc34fbe55f79f66e34f21916774a0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal\" (UID: \"b2bbc34fbe55f79f66e34f21916774a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal" Jul 08 21:32:57.560497 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.560399 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2bbc34fbe55f79f66e34f21916774a0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal\" (UID: \"b2bbc34fbe55f79f66e34f21916774a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal" Jul 08 21:32:57.560497 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.560418 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4a1b4940af681a5e34bbf5436c61df86-config\") pod \"kube-apiserver-proxy-ip-10-0-139-229.ec2.internal\" (UID: \"4a1b4940af681a5e34bbf5436c61df86\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-229.ec2.internal" Jul 08 21:32:57.560497 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.560457 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2bbc34fbe55f79f66e34f21916774a0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal\" (UID: \"b2bbc34fbe55f79f66e34f21916774a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal" Jul 08 21:32:57.560497 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.560459 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b2bbc34fbe55f79f66e34f21916774a0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal\" (UID: \"b2bbc34fbe55f79f66e34f21916774a0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal" Jul 08 21:32:57.560640 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.560495 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4a1b4940af681a5e34bbf5436c61df86-config\") pod \"kube-apiserver-proxy-ip-10-0-139-229.ec2.internal\" (UID: \"4a1b4940af681a5e34bbf5436c61df86\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-229.ec2.internal" Jul 08 21:32:57.588466 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.588447 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-229.ec2.internal\" not found" Jul 08 21:32:57.618637 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.618606 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal" Jul 08 21:32:57.624046 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:57.623564 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-229.ec2.internal" Jul 08 21:32:57.689395 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:57.689366 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-229.ec2.internal\" not found" Jul 08 21:32:58.838608 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:58.838548 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-229.ec2.internal\" not found" Jul 08 21:32:58.939042 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:58.939017 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-229.ec2.internal\" not found" Jul 08 21:32:58.945212 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:58.945195 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:32:59.007144 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.007119 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal" Jul 08 21:32:59.019129 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.019105 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 21:32:59.021130 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.021117 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-229.ec2.internal" Jul 08 21:32:59.029789 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.029770 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 21:32:59.047647 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.047628 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 21:32:59.047765 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.047725 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 21:32:59.047824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.047777 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 21:32:59.086951 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.086924 2580 apiserver.go:52] "Watching apiserver" Jul 08 21:32:59.105789 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.105728 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 21:32:59.107888 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.107286 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-jbjl2","kube-system/kube-apiserver-proxy-ip-10-0-139-229.ec2.internal","openshift-image-registry/node-ca-vpcft","openshift-multus/multus-additional-cni-plugins-9p7zn","openshift-multus/multus-vlzqf","openshift-multus/network-metrics-daemon-8bgg4","openshift-network-diagnostics/network-check-target-wlk2h","openshift-ovn-kubernetes/ovnkube-node-czfxp","kube-system/konnectivity-agent-bcglp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht","openshift-cluster-node-tuning-operator/tuned-h7h7c","openshift-dns/node-resolver-gdk2v","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal","openshift-network-operator/iptables-alerter-nt658"] Jul 08 21:32:59.107888 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.107788 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 21:32:59.109713 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.109692 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:32:59.109822 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:59.109785 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:32:59.111495 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.111479 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9gck\" (UniqueName: \"kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck\") pod \"network-check-target-wlk2h\" (UID: \"1dc458d1-00fb-427f-b71b-612cab976ce1\") " pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:32:59.112395 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.112379 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.114844 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.114822 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 21:32:59.114935 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.114825 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 21:32:59.114935 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.114858 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-m6kjl\"" Jul 08 21:32:59.115061 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.115049 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vpcft" Jul 08 21:32:59.117074 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.117047 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.117205 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.117141 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 21:32:59.117369 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.117349 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 21:32:59.117451 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.117387 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 21:32:59.117451 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.117418 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dv2jj\"" Jul 08 21:32:59.119249 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.119232 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.119621 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.119608 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 21:32:59.119687 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.119618 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 21:32:59.119742 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.119736 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 21:32:59.119825 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.119798 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 21:32:59.119970 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.119954 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 21:32:59.120013 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.119995 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m498v\"" Jul 08 21:32:59.120452 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.120438 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 21:32:59.121414 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.121370 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:32:59.121516 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:59.121430 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:32:59.121516 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.121456 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 21:32:59.121516 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.121498 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-m6z56\"" Jul 08 21:32:59.124117 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.124098 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bcglp" Jul 08 21:32:59.126267 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.126251 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 21:32:59.126335 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.126306 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 21:32:59.126411 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.126395 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-f6mpv\"" Jul 08 21:32:59.126484 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.126469 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nt658" Jul 08 21:32:59.128747 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.128733 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 21:32:59.129019 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.128998 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 21:32:59.129117 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.129074 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.129969 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.129949 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 21:32:59.130073 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.129949 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dxhlp\"" Jul 08 21:32:59.131846 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.131438 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 21:32:59.131846 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.131453 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 21:32:59.131846 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.131528 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.132428 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.132408 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 21:32:59.132428 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.132427 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 21:32:59.132568 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.132451 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-g9drp\"" Jul 08 21:32:59.132568 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.132417 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 21:32:59.132568 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.132511 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 21:32:59.133725 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.133701 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 21:32:59.133824 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.133795 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:32:59.134051 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.133864 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 21:32:59.134051 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:59.133868 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:32:59.134051 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.134002 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5s4m9\"" Jul 08 21:32:59.134051 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.134007 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 21:32:59.136426 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.136408 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gdk2v" Jul 08 21:32:59.139016 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.139002 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 21:32:59.139016 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.139011 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 21:32:59.139127 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.139057 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5dg8n\"" Jul 08 21:32:59.148302 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.148253 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 21:27:58 +0000 UTC" deadline="2028-02-21 06:40:13.208517179 +0000 UTC" Jul 08 21:32:59.148302 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.148301 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14217h7m14.06021894s" Jul 08 21:32:59.151686 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.151670 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-svt4h" Jul 08 21:32:59.157475 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.157460 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-svt4h" Jul 08 21:32:59.188953 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:59.188927 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2bbc34fbe55f79f66e34f21916774a0.slice/crio-e7b71b305c607fe44c78938a694e234b1eed04756dac20ba7502314cae71f686 WatchSource:0}: Error finding container e7b71b305c607fe44c78938a694e234b1eed04756dac20ba7502314cae71f686: Status 404 returned error can't find the container with id e7b71b305c607fe44c78938a694e234b1eed04756dac20ba7502314cae71f686 Jul 08 21:32:59.190601 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:59.190416 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a1b4940af681a5e34bbf5436c61df86.slice/crio-18d808d097e1334a42e19610ec8f688ec20a120314eaf36061fedea4276e9d76 WatchSource:0}: Error finding container 18d808d097e1334a42e19610ec8f688ec20a120314eaf36061fedea4276e9d76: Status 404 returned error can't find the container with id 18d808d097e1334a42e19610ec8f688ec20a120314eaf36061fedea4276e9d76 Jul 08 21:32:59.195915 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.195892 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 21:32:59.208072 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.208054 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 21:32:59.212012 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.211995 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-etc-openvswitch\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.212085 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212020 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.212085 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212046 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/159d4a44-ebf1-4d5e-913f-321bb9b767f8-ovn-node-metrics-cert\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.212195 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212088 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-host\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.212195 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212114 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e4803b19-4882-41f4-98a7-129d0493c630-host\") pod \"node-ca-vpcft\" (UID: \"e4803b19-4882-41f4-98a7-129d0493c630\") " pod="openshift-image-registry/node-ca-vpcft" Jul 08 21:32:59.212195 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212130 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.212195 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212144 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-kubelet\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.212195 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212158 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-run-netns\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.212195 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212177 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-cni-bin\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.212357 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212203 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg7hs\" (UniqueName: \"kubernetes.io/projected/b068ba38-cb80-4fa3-b474-294c4d2876a9-kube-api-access-zg7hs\") pod \"node-resolver-gdk2v\" (UID: \"b068ba38-cb80-4fa3-b474-294c4d2876a9\") " pod="openshift-dns/node-resolver-gdk2v" Jul 08 21:32:59.212357 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212219 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-etc-kubernetes\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.212357 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212241 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-var-lib-kubelet\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.212357 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212263 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/09c797d3-78e5-472e-b000-c66f6075ba77-cni-binary-copy\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.212357 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212277 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-multus-socket-dir-parent\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.212357 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212295 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-sys\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.212357 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212313 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-device-dir\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.212357 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212343 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1eac0252-00ff-4edd-a198-158b8a99ba18-kubelet-config\") pod \"global-pull-secret-syncer-jbjl2\" (UID: \"1eac0252-00ff-4edd-a198-158b8a99ba18\") " pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:32:59.212357 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212357 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-run\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.212623 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212370 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-lib-modules\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.212623 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212388 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsp8c\" (UniqueName: \"kubernetes.io/projected/7c115713-6b07-43e3-bcba-349ca1c338ef-kube-api-access-xsp8c\") pod \"iptables-alerter-nt658\" (UID: \"7c115713-6b07-43e3-bcba-349ca1c338ef\") " pod="openshift-network-operator/iptables-alerter-nt658" Jul 08 21:32:59.212623 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212410 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-multus-cni-dir\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.212623 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212432 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/09c797d3-78e5-472e-b000-c66f6075ba77-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.212623 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212461 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/736f94ce-c503-42ea-876b-b23b90389e46-tmp\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.212623 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212481 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7c115713-6b07-43e3-bcba-349ca1c338ef-host-slash\") pod \"iptables-alerter-nt658\" (UID: \"7c115713-6b07-43e3-bcba-349ca1c338ef\") " pod="openshift-network-operator/iptables-alerter-nt658" Jul 08 21:32:59.212623 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212498 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsx99\" (UniqueName: \"kubernetes.io/projected/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-kube-api-access-xsx99\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.212623 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdkwv\" (UniqueName: \"kubernetes.io/projected/dcc5996d-a926-4802-853f-c4991d8ecbef-kube-api-access-gdkwv\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.212623 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212525 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-etc-sysctl-conf\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.212623 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212544 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76pz4\" (UniqueName: \"kubernetes.io/projected/09c797d3-78e5-472e-b000-c66f6075ba77-kube-api-access-76pz4\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.212623 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212593 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-hostroot\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.213155 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212630 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/159d4a44-ebf1-4d5e-913f-321bb9b767f8-ovnkube-script-lib\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.213155 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212655 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b068ba38-cb80-4fa3-b474-294c4d2876a9-tmp-dir\") pod \"node-resolver-gdk2v\" (UID: \"b068ba38-cb80-4fa3-b474-294c4d2876a9\") " pod="openshift-dns/node-resolver-gdk2v" Jul 08 21:32:59.213155 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212679 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbj2z\" (UniqueName: \"kubernetes.io/projected/57804798-9f60-42ad-82f9-8ff4065af152-kube-api-access-jbj2z\") pod \"network-metrics-daemon-8bgg4\" (UID: \"57804798-9f60-42ad-82f9-8ff4065af152\") " pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:32:59.213155 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212702 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/990847bf-9c74-4725-84b0-fb3c2f05a5c2-konnectivity-ca\") pod \"konnectivity-agent-bcglp\" (UID: \"990847bf-9c74-4725-84b0-fb3c2f05a5c2\") " pod="kube-system/konnectivity-agent-bcglp" Jul 08 21:32:59.213155 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212724 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-sys-fs\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.213155 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212746 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-run-systemd\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.213155 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212767 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-cni-netd\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.213155 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212789 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-etc-sysconfig\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.213155 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212843 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-etc-systemd\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.213155 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212865 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/159d4a44-ebf1-4d5e-913f-321bb9b767f8-ovnkube-config\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.213155 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212889 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09c797d3-78e5-472e-b000-c66f6075ba77-system-cni-dir\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.213155 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212910 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/09c797d3-78e5-472e-b000-c66f6075ba77-cnibin\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.213155 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212939 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-socket-dir\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.213155 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212962 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-run-ovn\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.213155 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.212985 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-log-socket\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.213155 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213007 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-etc-modprobe-d\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.213761 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213036 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e4803b19-4882-41f4-98a7-129d0493c630-serviceca\") pod \"node-ca-vpcft\" (UID: \"e4803b19-4882-41f4-98a7-129d0493c630\") " pod="openshift-image-registry/node-ca-vpcft" Jul 08 21:32:59.213761 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213059 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/09c797d3-78e5-472e-b000-c66f6075ba77-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.213761 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213083 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/990847bf-9c74-4725-84b0-fb3c2f05a5c2-agent-certs\") pod \"konnectivity-agent-bcglp\" (UID: \"990847bf-9c74-4725-84b0-fb3c2f05a5c2\") " pod="kube-system/konnectivity-agent-bcglp" Jul 08 21:32:59.213761 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213106 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7c115713-6b07-43e3-bcba-349ca1c338ef-iptables-alerter-script\") pod \"iptables-alerter-nt658\" (UID: \"7c115713-6b07-43e3-bcba-349ca1c338ef\") " pod="openshift-network-operator/iptables-alerter-nt658" Jul 08 21:32:59.213761 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213127 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-registration-dir\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.213761 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213171 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-etc-selinux\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.213761 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213224 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-host-run-k8s-cni-cncf-io\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.213761 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213252 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/736f94ce-c503-42ea-876b-b23b90389e46-etc-tuned\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.213761 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213286 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-host-run-multus-certs\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.213761 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213321 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-systemd-units\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.213761 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213355 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-slash\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.213761 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213378 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-run-ovn-kubernetes\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.213761 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213399 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1eac0252-00ff-4edd-a198-158b8a99ba18-dbus\") pod \"global-pull-secret-syncer-jbjl2\" (UID: \"1eac0252-00ff-4edd-a198-158b8a99ba18\") " pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:32:59.213761 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213416 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-system-cni-dir\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.213761 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213430 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dcc5996d-a926-4802-853f-c4991d8ecbef-cni-binary-copy\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.213761 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213446 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-multus-conf-dir\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.214354 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213460 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-node-log\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.214354 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213491 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b068ba38-cb80-4fa3-b474-294c4d2876a9-hosts-file\") pod \"node-resolver-gdk2v\" (UID: \"b068ba38-cb80-4fa3-b474-294c4d2876a9\") " pod="openshift-dns/node-resolver-gdk2v" Jul 08 21:32:59.214354 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/09c797d3-78e5-472e-b000-c66f6075ba77-os-release\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.214354 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213527 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4j88\" (UniqueName: \"kubernetes.io/projected/736f94ce-c503-42ea-876b-b23b90389e46-kube-api-access-q4j88\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.214354 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213540 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-cnibin\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.214354 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213553 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-host-var-lib-cni-bin\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.214354 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213566 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-host-var-lib-cni-multus\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.214354 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213585 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dcc5996d-a926-4802-853f-c4991d8ecbef-multus-daemon-config\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.214354 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213599 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/159d4a44-ebf1-4d5e-913f-321bb9b767f8-env-overrides\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.214354 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213637 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9gck\" (UniqueName: \"kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck\") pod \"network-check-target-wlk2h\" (UID: \"1dc458d1-00fb-427f-b71b-612cab976ce1\") " pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:32:59.214354 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213719 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-etc-sysctl-d\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.214354 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213758 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjsrh\" (UniqueName: \"kubernetes.io/projected/e4803b19-4882-41f4-98a7-129d0493c630-kube-api-access-vjsrh\") pod \"node-ca-vpcft\" (UID: \"e4803b19-4882-41f4-98a7-129d0493c630\") " pod="openshift-image-registry/node-ca-vpcft" Jul 08 21:32:59.214354 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213789 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/09c797d3-78e5-472e-b000-c66f6075ba77-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.214354 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213829 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs\") pod \"network-metrics-daemon-8bgg4\" (UID: \"57804798-9f60-42ad-82f9-8ff4065af152\") " pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:32:59.214354 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213851 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-host-run-netns\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.214354 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213872 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-var-lib-openvswitch\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.214865 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213915 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-run-openvswitch\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.214865 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213956 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-os-release\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.214865 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.213992 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-etc-kubernetes\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.214865 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.214015 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt7hb\" (UniqueName: \"kubernetes.io/projected/159d4a44-ebf1-4d5e-913f-321bb9b767f8-kube-api-access-dt7hb\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.214865 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.214039 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret\") pod \"global-pull-secret-syncer-jbjl2\" (UID: \"1eac0252-00ff-4edd-a198-158b8a99ba18\") " pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:32:59.214865 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.214064 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-host-var-lib-kubelet\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.220294 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:59.220280 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:32:59.220294 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:59.220296 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:32:59.220393 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:59.220305 2580 projected.go:196] Error preparing data for projected volume kube-api-access-j9gck for pod openshift-network-diagnostics/network-check-target-wlk2h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:32:59.220393 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:59.220348 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck podName:1dc458d1-00fb-427f-b71b-612cab976ce1 nodeName:}" failed. No retries permitted until 2026-07-08 21:32:59.720334116 +0000 UTC m=+2.018219028 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-j9gck" (UniqueName: "kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck") pod "network-check-target-wlk2h" (UID: "1dc458d1-00fb-427f-b71b-612cab976ce1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:32:59.239483 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.239436 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal" event={"ID":"b2bbc34fbe55f79f66e34f21916774a0","Type":"ContainerStarted","Data":"e7b71b305c607fe44c78938a694e234b1eed04756dac20ba7502314cae71f686"} Jul 08 21:32:59.240466 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.240445 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-229.ec2.internal" event={"ID":"4a1b4940af681a5e34bbf5436c61df86","Type":"ContainerStarted","Data":"18d808d097e1334a42e19610ec8f688ec20a120314eaf36061fedea4276e9d76"} Jul 08 21:32:59.240552 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.240531 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:32:59.314375 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314347 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-etc-sysctl-d\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.314375 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314374 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjsrh\" (UniqueName: \"kubernetes.io/projected/e4803b19-4882-41f4-98a7-129d0493c630-kube-api-access-vjsrh\") pod \"node-ca-vpcft\" (UID: \"e4803b19-4882-41f4-98a7-129d0493c630\") " pod="openshift-image-registry/node-ca-vpcft" Jul 08 21:32:59.314570 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314391 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/09c797d3-78e5-472e-b000-c66f6075ba77-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.314570 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314407 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs\") pod \"network-metrics-daemon-8bgg4\" (UID: \"57804798-9f60-42ad-82f9-8ff4065af152\") " pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:32:59.314570 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314424 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-host-run-netns\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.314570 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314478 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-host-run-netns\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.314570 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:59.314502 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:32:59.314570 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314537 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-var-lib-openvswitch\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.314884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314580 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-run-openvswitch\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.314884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314609 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-os-release\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.314884 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:59.314634 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs podName:57804798-9f60-42ad-82f9-8ff4065af152 nodeName:}" failed. No retries permitted until 2026-07-08 21:32:59.814615215 +0000 UTC m=+2.112500129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs") pod "network-metrics-daemon-8bgg4" (UID: "57804798-9f60-42ad-82f9-8ff4065af152") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:32:59.314884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314653 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-run-openvswitch\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.314884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314664 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-etc-kubernetes\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.314884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314662 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-var-lib-openvswitch\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.314884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314688 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt7hb\" (UniqueName: \"kubernetes.io/projected/159d4a44-ebf1-4d5e-913f-321bb9b767f8-kube-api-access-dt7hb\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.314884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314731 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-etc-sysctl-d\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.314884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314741 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret\") pod \"global-pull-secret-syncer-jbjl2\" (UID: \"1eac0252-00ff-4edd-a198-158b8a99ba18\") " pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:32:59.314884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314772 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-os-release\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.314884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314783 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-host-var-lib-kubelet\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.314884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314837 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-etc-kubernetes\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.314884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314840 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-etc-openvswitch\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.314884 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:59.314876 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:32:59.314884 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314879 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.315575 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314907 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-host-var-lib-kubelet\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.315575 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314885 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-etc-openvswitch\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.315575 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:59.314925 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret podName:1eac0252-00ff-4edd-a198-158b8a99ba18 nodeName:}" failed. No retries permitted until 2026-07-08 21:32:59.814909108 +0000 UTC m=+2.112794021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret") pod "global-pull-secret-syncer-jbjl2" (UID: "1eac0252-00ff-4edd-a198-158b8a99ba18") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:32:59.315575 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314922 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.315575 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314911 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/159d4a44-ebf1-4d5e-913f-321bb9b767f8-ovn-node-metrics-cert\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.315575 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314961 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-host\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.315575 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314976 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e4803b19-4882-41f4-98a7-129d0493c630-host\") pod \"node-ca-vpcft\" (UID: \"e4803b19-4882-41f4-98a7-129d0493c630\") " pod="openshift-image-registry/node-ca-vpcft" Jul 08 21:32:59.315575 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.314999 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.315575 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315030 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-host\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.315575 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315032 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/09c797d3-78e5-472e-b000-c66f6075ba77-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.315575 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315062 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-kubelet\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.315575 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315069 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.315575 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315090 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-run-netns\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.315575 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315042 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e4803b19-4882-41f4-98a7-129d0493c630-host\") pod \"node-ca-vpcft\" (UID: \"e4803b19-4882-41f4-98a7-129d0493c630\") " pod="openshift-image-registry/node-ca-vpcft" Jul 08 21:32:59.315575 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315106 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-cni-bin\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.315575 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315121 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-kubelet\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.315575 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315128 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg7hs\" (UniqueName: \"kubernetes.io/projected/b068ba38-cb80-4fa3-b474-294c4d2876a9-kube-api-access-zg7hs\") pod \"node-resolver-gdk2v\" (UID: \"b068ba38-cb80-4fa3-b474-294c4d2876a9\") " pod="openshift-dns/node-resolver-gdk2v" Jul 08 21:32:59.316335 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315138 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-run-netns\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.316335 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315151 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-cni-bin\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.316335 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315151 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-etc-kubernetes\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.316335 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315191 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-var-lib-kubelet\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.316335 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315202 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 21:32:59.316335 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315214 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/09c797d3-78e5-472e-b000-c66f6075ba77-cni-binary-copy\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.316335 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315237 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-multus-socket-dir-parent\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.316335 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315243 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-var-lib-kubelet\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.316335 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315258 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-sys\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.316335 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315279 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-device-dir\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.316335 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315303 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1eac0252-00ff-4edd-a198-158b8a99ba18-kubelet-config\") pod \"global-pull-secret-syncer-jbjl2\" (UID: \"1eac0252-00ff-4edd-a198-158b8a99ba18\") " pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:32:59.316335 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315311 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-multus-socket-dir-parent\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.316335 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315326 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-run\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.316335 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315315 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-sys\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.316335 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315347 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-lib-modules\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.316335 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315354 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-device-dir\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.316335 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315368 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsp8c\" (UniqueName: \"kubernetes.io/projected/7c115713-6b07-43e3-bcba-349ca1c338ef-kube-api-access-xsp8c\") pod \"iptables-alerter-nt658\" (UID: \"7c115713-6b07-43e3-bcba-349ca1c338ef\") " pod="openshift-network-operator/iptables-alerter-nt658" Jul 08 21:32:59.316335 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315380 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-run\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.317352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315384 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1eac0252-00ff-4edd-a198-158b8a99ba18-kubelet-config\") pod \"global-pull-secret-syncer-jbjl2\" (UID: \"1eac0252-00ff-4edd-a198-158b8a99ba18\") " pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:32:59.317352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315425 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-multus-cni-dir\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.317352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315206 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-etc-kubernetes\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.317352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315453 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/09c797d3-78e5-472e-b000-c66f6075ba77-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.317352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315485 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-lib-modules\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.317352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315486 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/736f94ce-c503-42ea-876b-b23b90389e46-tmp\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.317352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315524 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-multus-cni-dir\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.317352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315556 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7c115713-6b07-43e3-bcba-349ca1c338ef-host-slash\") pod \"iptables-alerter-nt658\" (UID: \"7c115713-6b07-43e3-bcba-349ca1c338ef\") " pod="openshift-network-operator/iptables-alerter-nt658" Jul 08 21:32:59.317352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315582 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsx99\" (UniqueName: \"kubernetes.io/projected/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-kube-api-access-xsx99\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.317352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315604 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdkwv\" (UniqueName: \"kubernetes.io/projected/dcc5996d-a926-4802-853f-c4991d8ecbef-kube-api-access-gdkwv\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.317352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315607 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7c115713-6b07-43e3-bcba-349ca1c338ef-host-slash\") pod \"iptables-alerter-nt658\" (UID: \"7c115713-6b07-43e3-bcba-349ca1c338ef\") " pod="openshift-network-operator/iptables-alerter-nt658" Jul 08 21:32:59.317352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315637 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-etc-sysctl-conf\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.317352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315685 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76pz4\" (UniqueName: \"kubernetes.io/projected/09c797d3-78e5-472e-b000-c66f6075ba77-kube-api-access-76pz4\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.317352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315711 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-hostroot\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.317352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315752 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/159d4a44-ebf1-4d5e-913f-321bb9b767f8-ovnkube-script-lib\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.317352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315768 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/09c797d3-78e5-472e-b000-c66f6075ba77-cni-binary-copy\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.317352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315854 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-hostroot\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.318168 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315887 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b068ba38-cb80-4fa3-b474-294c4d2876a9-tmp-dir\") pod \"node-resolver-gdk2v\" (UID: \"b068ba38-cb80-4fa3-b474-294c4d2876a9\") " pod="openshift-dns/node-resolver-gdk2v" Jul 08 21:32:59.318168 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315912 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbj2z\" (UniqueName: \"kubernetes.io/projected/57804798-9f60-42ad-82f9-8ff4065af152-kube-api-access-jbj2z\") pod \"network-metrics-daemon-8bgg4\" (UID: \"57804798-9f60-42ad-82f9-8ff4065af152\") " pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:32:59.318168 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315939 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/990847bf-9c74-4725-84b0-fb3c2f05a5c2-konnectivity-ca\") pod \"konnectivity-agent-bcglp\" (UID: \"990847bf-9c74-4725-84b0-fb3c2f05a5c2\") " pod="kube-system/konnectivity-agent-bcglp" Jul 08 21:32:59.318168 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315941 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/09c797d3-78e5-472e-b000-c66f6075ba77-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.318168 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315951 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-etc-sysctl-conf\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.318168 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315962 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-sys-fs\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.318168 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.315987 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-run-systemd\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.318168 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316013 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-cni-netd\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.318168 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316037 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-etc-sysconfig\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.318168 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316058 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-etc-systemd\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.318168 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316082 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/159d4a44-ebf1-4d5e-913f-321bb9b767f8-ovnkube-config\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.318168 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316108 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09c797d3-78e5-472e-b000-c66f6075ba77-system-cni-dir\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.318168 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316133 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/09c797d3-78e5-472e-b000-c66f6075ba77-cnibin\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.318168 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316157 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-socket-dir\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.318168 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316182 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-run-ovn\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.318168 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316206 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-log-socket\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.318168 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316218 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-etc-sysconfig\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.318906 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316229 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-etc-modprobe-d\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.318906 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316255 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e4803b19-4882-41f4-98a7-129d0493c630-serviceca\") pod \"node-ca-vpcft\" (UID: \"e4803b19-4882-41f4-98a7-129d0493c630\") " pod="openshift-image-registry/node-ca-vpcft" Jul 08 21:32:59.318906 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316323 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-run-systemd\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.318906 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316357 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-cni-netd\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.318906 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316358 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-etc-modprobe-d\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.318906 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316377 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b068ba38-cb80-4fa3-b474-294c4d2876a9-tmp-dir\") pod \"node-resolver-gdk2v\" (UID: \"b068ba38-cb80-4fa3-b474-294c4d2876a9\") " pod="openshift-dns/node-resolver-gdk2v" Jul 08 21:32:59.318906 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316390 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-run-ovn\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.318906 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316422 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/09c797d3-78e5-472e-b000-c66f6075ba77-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.318906 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316447 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-socket-dir\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.318906 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316443 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-log-socket\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.318906 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316488 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09c797d3-78e5-472e-b000-c66f6075ba77-system-cni-dir\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.318906 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316448 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/990847bf-9c74-4725-84b0-fb3c2f05a5c2-agent-certs\") pod \"konnectivity-agent-bcglp\" (UID: \"990847bf-9c74-4725-84b0-fb3c2f05a5c2\") " pod="kube-system/konnectivity-agent-bcglp" Jul 08 21:32:59.318906 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316526 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7c115713-6b07-43e3-bcba-349ca1c338ef-iptables-alerter-script\") pod \"iptables-alerter-nt658\" (UID: \"7c115713-6b07-43e3-bcba-349ca1c338ef\") " pod="openshift-network-operator/iptables-alerter-nt658" Jul 08 21:32:59.318906 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316532 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/736f94ce-c503-42ea-876b-b23b90389e46-etc-systemd\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.318906 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316491 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/09c797d3-78e5-472e-b000-c66f6075ba77-cnibin\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.318906 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316554 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-registration-dir\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.318906 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316486 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/990847bf-9c74-4725-84b0-fb3c2f05a5c2-konnectivity-ca\") pod \"konnectivity-agent-bcglp\" (UID: \"990847bf-9c74-4725-84b0-fb3c2f05a5c2\") " pod="kube-system/konnectivity-agent-bcglp" Jul 08 21:32:59.319449 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316597 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-etc-selinux\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.319449 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316624 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-host-run-k8s-cni-cncf-io\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.319449 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316638 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/09c797d3-78e5-472e-b000-c66f6075ba77-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.319449 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316665 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e4803b19-4882-41f4-98a7-129d0493c630-serviceca\") pod \"node-ca-vpcft\" (UID: \"e4803b19-4882-41f4-98a7-129d0493c630\") " pod="openshift-image-registry/node-ca-vpcft" Jul 08 21:32:59.319449 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316686 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/736f94ce-c503-42ea-876b-b23b90389e46-etc-tuned\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.319449 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316716 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-host-run-multus-certs\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.319449 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316719 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-sys-fs\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.319449 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316740 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-systemd-units\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.319449 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316785 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-host-run-multus-certs\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.319449 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316791 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-slash\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.319449 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316839 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-run-ovn-kubernetes\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.319449 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316866 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1eac0252-00ff-4edd-a198-158b8a99ba18-dbus\") pod \"global-pull-secret-syncer-jbjl2\" (UID: \"1eac0252-00ff-4edd-a198-158b8a99ba18\") " pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:32:59.319449 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316906 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-system-cni-dir\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.319449 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316931 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dcc5996d-a926-4802-853f-c4991d8ecbef-cni-binary-copy\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.319449 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316958 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-multus-conf-dir\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.319449 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316971 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/159d4a44-ebf1-4d5e-913f-321bb9b767f8-ovnkube-config\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.319449 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.316983 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-node-log\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.319449 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317007 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b068ba38-cb80-4fa3-b474-294c4d2876a9-hosts-file\") pod \"node-resolver-gdk2v\" (UID: \"b068ba38-cb80-4fa3-b474-294c4d2876a9\") " pod="openshift-dns/node-resolver-gdk2v" Jul 08 21:32:59.319987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317029 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-run-ovn-kubernetes\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.319987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317055 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7c115713-6b07-43e3-bcba-349ca1c338ef-iptables-alerter-script\") pod \"iptables-alerter-nt658\" (UID: \"7c115713-6b07-43e3-bcba-349ca1c338ef\") " pod="openshift-network-operator/iptables-alerter-nt658" Jul 08 21:32:59.319987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317076 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/09c797d3-78e5-472e-b000-c66f6075ba77-os-release\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.319987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317085 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-registration-dir\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.319987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317103 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-node-log\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.319987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317122 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4j88\" (UniqueName: \"kubernetes.io/projected/736f94ce-c503-42ea-876b-b23b90389e46-kube-api-access-q4j88\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.319987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317148 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b068ba38-cb80-4fa3-b474-294c4d2876a9-hosts-file\") pod \"node-resolver-gdk2v\" (UID: \"b068ba38-cb80-4fa3-b474-294c4d2876a9\") " pod="openshift-dns/node-resolver-gdk2v" Jul 08 21:32:59.319987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317165 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-cnibin\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.319987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317188 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1eac0252-00ff-4edd-a198-158b8a99ba18-dbus\") pod \"global-pull-secret-syncer-jbjl2\" (UID: \"1eac0252-00ff-4edd-a198-158b8a99ba18\") " pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:32:59.319987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317190 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-system-cni-dir\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.319987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317221 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-host-var-lib-cni-bin\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.319987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317272 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-etc-selinux\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.319987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317290 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-host-run-k8s-cni-cncf-io\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.319987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317340 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-host-slash\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.319987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317388 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/09c797d3-78e5-472e-b000-c66f6075ba77-os-release\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.319987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317422 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-host-var-lib-cni-bin\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.319987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317456 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/159d4a44-ebf1-4d5e-913f-321bb9b767f8-systemd-units\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.319987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317413 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-host-var-lib-cni-multus\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.320460 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317505 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-cnibin\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.320460 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317511 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dcc5996d-a926-4802-853f-c4991d8ecbef-multus-daemon-config\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.320460 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317660 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/159d4a44-ebf1-4d5e-913f-321bb9b767f8-env-overrides\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.320460 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317685 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dcc5996d-a926-4802-853f-c4991d8ecbef-cni-binary-copy\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.320460 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317710 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-multus-conf-dir\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.320460 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.317733 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dcc5996d-a926-4802-853f-c4991d8ecbef-host-var-lib-cni-multus\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.320460 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.318128 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/159d4a44-ebf1-4d5e-913f-321bb9b767f8-ovnkube-script-lib\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.320460 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.318168 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dcc5996d-a926-4802-853f-c4991d8ecbef-multus-daemon-config\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.320460 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.318186 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/159d4a44-ebf1-4d5e-913f-321bb9b767f8-env-overrides\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.320460 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.318788 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/736f94ce-c503-42ea-876b-b23b90389e46-tmp\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.320460 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.318978 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/736f94ce-c503-42ea-876b-b23b90389e46-etc-tuned\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.320460 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.319032 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/159d4a44-ebf1-4d5e-913f-321bb9b767f8-ovn-node-metrics-cert\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.320460 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.319115 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/990847bf-9c74-4725-84b0-fb3c2f05a5c2-agent-certs\") pod \"konnectivity-agent-bcglp\" (UID: \"990847bf-9c74-4725-84b0-fb3c2f05a5c2\") " pod="kube-system/konnectivity-agent-bcglp" Jul 08 21:32:59.324509 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.324474 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjsrh\" (UniqueName: \"kubernetes.io/projected/e4803b19-4882-41f4-98a7-129d0493c630-kube-api-access-vjsrh\") pod \"node-ca-vpcft\" (UID: \"e4803b19-4882-41f4-98a7-129d0493c630\") " pod="openshift-image-registry/node-ca-vpcft" Jul 08 21:32:59.327390 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.327371 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt7hb\" (UniqueName: \"kubernetes.io/projected/159d4a44-ebf1-4d5e-913f-321bb9b767f8-kube-api-access-dt7hb\") pod \"ovnkube-node-czfxp\" (UID: \"159d4a44-ebf1-4d5e-913f-321bb9b767f8\") " pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.329911 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.329881 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdkwv\" (UniqueName: \"kubernetes.io/projected/dcc5996d-a926-4802-853f-c4991d8ecbef-kube-api-access-gdkwv\") pod \"multus-vlzqf\" (UID: \"dcc5996d-a926-4802-853f-c4991d8ecbef\") " pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.330203 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.330175 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsp8c\" (UniqueName: \"kubernetes.io/projected/7c115713-6b07-43e3-bcba-349ca1c338ef-kube-api-access-xsp8c\") pod \"iptables-alerter-nt658\" (UID: \"7c115713-6b07-43e3-bcba-349ca1c338ef\") " pod="openshift-network-operator/iptables-alerter-nt658" Jul 08 21:32:59.330430 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.330407 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4j88\" (UniqueName: \"kubernetes.io/projected/736f94ce-c503-42ea-876b-b23b90389e46-kube-api-access-q4j88\") pod \"tuned-h7h7c\" (UID: \"736f94ce-c503-42ea-876b-b23b90389e46\") " pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.330569 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.330546 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg7hs\" (UniqueName: \"kubernetes.io/projected/b068ba38-cb80-4fa3-b474-294c4d2876a9-kube-api-access-zg7hs\") pod \"node-resolver-gdk2v\" (UID: \"b068ba38-cb80-4fa3-b474-294c4d2876a9\") " pod="openshift-dns/node-resolver-gdk2v" Jul 08 21:32:59.331074 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.331054 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbj2z\" (UniqueName: \"kubernetes.io/projected/57804798-9f60-42ad-82f9-8ff4065af152-kube-api-access-jbj2z\") pod \"network-metrics-daemon-8bgg4\" (UID: \"57804798-9f60-42ad-82f9-8ff4065af152\") " pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:32:59.331345 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.331327 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsx99\" (UniqueName: \"kubernetes.io/projected/a17f0dcb-0507-43d8-8a2d-b313d8fad47f-kube-api-access-xsx99\") pod \"aws-ebs-csi-driver-node-mp2ht\" (UID: \"a17f0dcb-0507-43d8-8a2d-b313d8fad47f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.331430 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.331385 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76pz4\" (UniqueName: \"kubernetes.io/projected/09c797d3-78e5-472e-b000-c66f6075ba77-kube-api-access-76pz4\") pod \"multus-additional-cni-plugins-9p7zn\" (UID: \"09c797d3-78e5-472e-b000-c66f6075ba77\") " pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.437053 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.436980 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" Jul 08 21:32:59.447316 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:59.447295 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod736f94ce_c503_42ea_876b_b23b90389e46.slice/crio-9b8cf7a7a84f01777076dcdc7f63e6a763ca492f764c906ad9da9bf365df5133 WatchSource:0}: Error finding container 9b8cf7a7a84f01777076dcdc7f63e6a763ca492f764c906ad9da9bf365df5133: Status 404 returned error can't find the container with id 9b8cf7a7a84f01777076dcdc7f63e6a763ca492f764c906ad9da9bf365df5133 Jul 08 21:32:59.450255 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.450235 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vpcft" Jul 08 21:32:59.457991 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:59.457971 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4803b19_4882_41f4_98a7_129d0493c630.slice/crio-67668a60cf6fc1cc57dd19faa1027e3610800a7c0adbbf4b8151cd4539242ca9 WatchSource:0}: Error finding container 67668a60cf6fc1cc57dd19faa1027e3610800a7c0adbbf4b8151cd4539242ca9: Status 404 returned error can't find the container with id 67668a60cf6fc1cc57dd19faa1027e3610800a7c0adbbf4b8151cd4539242ca9 Jul 08 21:32:59.475621 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.475603 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:32:59.477433 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.477407 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9p7zn" Jul 08 21:32:59.485791 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:59.485771 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09c797d3_78e5_472e_b000_c66f6075ba77.slice/crio-7940d51869abf18ff5b4aac271885636c13590fcee40c6227af4f1a02db85fae WatchSource:0}: Error finding container 7940d51869abf18ff5b4aac271885636c13590fcee40c6227af4f1a02db85fae: Status 404 returned error can't find the container with id 7940d51869abf18ff5b4aac271885636c13590fcee40c6227af4f1a02db85fae Jul 08 21:32:59.496113 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.496094 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vlzqf" Jul 08 21:32:59.501883 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.501861 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bcglp" Jul 08 21:32:59.505594 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:59.505571 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcc5996d_a926_4802_853f_c4991d8ecbef.slice/crio-2f4435181a5947ff24eb8c9f35df0e7630bae82607b9e7512ac089c297fd57e3 WatchSource:0}: Error finding container 2f4435181a5947ff24eb8c9f35df0e7630bae82607b9e7512ac089c297fd57e3: Status 404 returned error can't find the container with id 2f4435181a5947ff24eb8c9f35df0e7630bae82607b9e7512ac089c297fd57e3 Jul 08 21:32:59.508558 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.508536 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nt658" Jul 08 21:32:59.510056 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:59.510037 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod990847bf_9c74_4725_84b0_fb3c2f05a5c2.slice/crio-106a4ab090380599ec7a3abd426902932587f74207a12777380f362aaa309b86 WatchSource:0}: Error finding container 106a4ab090380599ec7a3abd426902932587f74207a12777380f362aaa309b86: Status 404 returned error can't find the container with id 106a4ab090380599ec7a3abd426902932587f74207a12777380f362aaa309b86 Jul 08 21:32:59.514516 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.514500 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:32:59.516994 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:59.516977 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c115713_6b07_43e3_bcba_349ca1c338ef.slice/crio-126ca2913a0b729fdc880869d610bd56558ec2aac8af1af622572eab0937c3c8 WatchSource:0}: Error finding container 126ca2913a0b729fdc880869d610bd56558ec2aac8af1af622572eab0937c3c8: Status 404 returned error can't find the container with id 126ca2913a0b729fdc880869d610bd56558ec2aac8af1af622572eab0937c3c8 Jul 08 21:32:59.519082 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.519061 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" Jul 08 21:32:59.524042 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:59.524020 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod159d4a44_ebf1_4d5e_913f_321bb9b767f8.slice/crio-e87b1b53142b405c07b67a2bff63c4fd4fc64e5532ad450ea6bc17c7b6f5c54c WatchSource:0}: Error finding container e87b1b53142b405c07b67a2bff63c4fd4fc64e5532ad450ea6bc17c7b6f5c54c: Status 404 returned error can't find the container with id e87b1b53142b405c07b67a2bff63c4fd4fc64e5532ad450ea6bc17c7b6f5c54c Jul 08 21:32:59.524135 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.524052 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gdk2v" Jul 08 21:32:59.528835 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:59.528776 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda17f0dcb_0507_43d8_8a2d_b313d8fad47f.slice/crio-7477855f5d85a2b20ff04b7e8acacb5ccd20f2f25c35035914edd6719269c114 WatchSource:0}: Error finding container 7477855f5d85a2b20ff04b7e8acacb5ccd20f2f25c35035914edd6719269c114: Status 404 returned error can't find the container with id 7477855f5d85a2b20ff04b7e8acacb5ccd20f2f25c35035914edd6719269c114 Jul 08 21:32:59.534331 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:32:59.534312 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb068ba38_cb80_4fa3_b474_294c4d2876a9.slice/crio-813553f6d4a88b04b093a22da9eab28f40b96c8ed9587d47889f2b44baac86de WatchSource:0}: Error finding container 813553f6d4a88b04b093a22da9eab28f40b96c8ed9587d47889f2b44baac86de: Status 404 returned error can't find the container with id 813553f6d4a88b04b093a22da9eab28f40b96c8ed9587d47889f2b44baac86de Jul 08 21:32:59.821542 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.821434 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9gck\" (UniqueName: \"kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck\") pod \"network-check-target-wlk2h\" (UID: \"1dc458d1-00fb-427f-b71b-612cab976ce1\") " pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:32:59.821542 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.821498 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs\") pod \"network-metrics-daemon-8bgg4\" (UID: \"57804798-9f60-42ad-82f9-8ff4065af152\") " pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:32:59.821542 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:32:59.821529 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret\") pod \"global-pull-secret-syncer-jbjl2\" (UID: \"1eac0252-00ff-4edd-a198-158b8a99ba18\") " pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:32:59.821823 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:59.821642 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:32:59.821823 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:59.821689 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret podName:1eac0252-00ff-4edd-a198-158b8a99ba18 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:00.821676035 +0000 UTC m=+3.119560948 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret") pod "global-pull-secret-syncer-jbjl2" (UID: "1eac0252-00ff-4edd-a198-158b8a99ba18") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:32:59.822096 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:59.822076 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:32:59.822165 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:59.822103 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:32:59.822165 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:59.822117 2580 projected.go:196] Error preparing data for projected volume kube-api-access-j9gck for pod openshift-network-diagnostics/network-check-target-wlk2h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:32:59.822165 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:59.822161 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck podName:1dc458d1-00fb-427f-b71b-612cab976ce1 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:00.822146371 +0000 UTC m=+3.120031290 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9gck" (UniqueName: "kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck") pod "network-check-target-wlk2h" (UID: "1dc458d1-00fb-427f-b71b-612cab976ce1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:32:59.822321 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:59.822227 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:32:59.822321 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:32:59.822258 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs podName:57804798-9f60-42ad-82f9-8ff4065af152 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:00.822246997 +0000 UTC m=+3.120131913 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs") pod "network-metrics-daemon-8bgg4" (UID: "57804798-9f60-42ad-82f9-8ff4065af152") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:00.160890 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:00.160533 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 21:27:59 +0000 UTC" deadline="2028-02-12 06:07:08.384281907 +0000 UTC" Jul 08 21:33:00.160890 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:00.160841 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14000h34m8.223447289s" Jul 08 21:33:00.239422 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:00.239376 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:00.239594 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:00.239526 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:00.240011 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:00.239977 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:00.240126 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:00.240107 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:00.268310 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:00.268271 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" event={"ID":"a17f0dcb-0507-43d8-8a2d-b313d8fad47f","Type":"ContainerStarted","Data":"7477855f5d85a2b20ff04b7e8acacb5ccd20f2f25c35035914edd6719269c114"} Jul 08 21:33:00.298520 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:00.298419 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bcglp" event={"ID":"990847bf-9c74-4725-84b0-fb3c2f05a5c2","Type":"ContainerStarted","Data":"106a4ab090380599ec7a3abd426902932587f74207a12777380f362aaa309b86"} Jul 08 21:33:00.310826 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:00.310725 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9p7zn" event={"ID":"09c797d3-78e5-472e-b000-c66f6075ba77","Type":"ContainerStarted","Data":"7940d51869abf18ff5b4aac271885636c13590fcee40c6227af4f1a02db85fae"} Jul 08 21:33:00.320142 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:00.319986 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vpcft" event={"ID":"e4803b19-4882-41f4-98a7-129d0493c630","Type":"ContainerStarted","Data":"67668a60cf6fc1cc57dd19faa1027e3610800a7c0adbbf4b8151cd4539242ca9"} Jul 08 21:33:00.335771 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:00.335737 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" event={"ID":"736f94ce-c503-42ea-876b-b23b90389e46","Type":"ContainerStarted","Data":"9b8cf7a7a84f01777076dcdc7f63e6a763ca492f764c906ad9da9bf365df5133"} Jul 08 21:33:00.358023 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:00.357988 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gdk2v" event={"ID":"b068ba38-cb80-4fa3-b474-294c4d2876a9","Type":"ContainerStarted","Data":"813553f6d4a88b04b093a22da9eab28f40b96c8ed9587d47889f2b44baac86de"} Jul 08 21:33:00.371218 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:00.371183 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" event={"ID":"159d4a44-ebf1-4d5e-913f-321bb9b767f8","Type":"ContainerStarted","Data":"e87b1b53142b405c07b67a2bff63c4fd4fc64e5532ad450ea6bc17c7b6f5c54c"} Jul 08 21:33:00.378382 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:00.378353 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nt658" event={"ID":"7c115713-6b07-43e3-bcba-349ca1c338ef","Type":"ContainerStarted","Data":"126ca2913a0b729fdc880869d610bd56558ec2aac8af1af622572eab0937c3c8"} Jul 08 21:33:00.382642 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:00.382615 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vlzqf" event={"ID":"dcc5996d-a926-4802-853f-c4991d8ecbef","Type":"ContainerStarted","Data":"2f4435181a5947ff24eb8c9f35df0e7630bae82607b9e7512ac089c297fd57e3"} Jul 08 21:33:00.554981 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:00.554907 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:33:00.828860 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:00.828760 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9gck\" (UniqueName: \"kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck\") pod \"network-check-target-wlk2h\" (UID: \"1dc458d1-00fb-427f-b71b-612cab976ce1\") " pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:00.828860 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:00.828831 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs\") pod \"network-metrics-daemon-8bgg4\" (UID: \"57804798-9f60-42ad-82f9-8ff4065af152\") " pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:00.829057 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:00.828864 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret\") pod \"global-pull-secret-syncer-jbjl2\" (UID: \"1eac0252-00ff-4edd-a198-158b8a99ba18\") " pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:00.829057 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:00.829008 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:33:00.829153 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:00.829072 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret podName:1eac0252-00ff-4edd-a198-158b8a99ba18 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:02.829053249 +0000 UTC m=+5.126938166 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret") pod "global-pull-secret-syncer-jbjl2" (UID: "1eac0252-00ff-4edd-a198-158b8a99ba18") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:33:00.829738 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:00.829618 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:33:00.829738 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:00.829638 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:33:00.829738 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:00.829651 2580 projected.go:196] Error preparing data for projected volume kube-api-access-j9gck for pod openshift-network-diagnostics/network-check-target-wlk2h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:00.829738 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:00.829694 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck podName:1dc458d1-00fb-427f-b71b-612cab976ce1 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:02.829680176 +0000 UTC m=+5.127565097 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9gck" (UniqueName: "kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck") pod "network-check-target-wlk2h" (UID: "1dc458d1-00fb-427f-b71b-612cab976ce1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:00.829985 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:00.829758 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:00.829985 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:00.829791 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs podName:57804798-9f60-42ad-82f9-8ff4065af152 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:02.829780573 +0000 UTC m=+5.127665489 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs") pod "network-metrics-daemon-8bgg4" (UID: "57804798-9f60-42ad-82f9-8ff4065af152") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:01.161579 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:01.161493 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 21:27:59 +0000 UTC" deadline="2028-01-24 13:58:10.140350748 +0000 UTC" Jul 08 21:33:01.161579 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:01.161530 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13552h25m8.978825326s" Jul 08 21:33:01.236842 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:01.236773 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:01.237031 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:01.236974 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:01.520221 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:01.519899 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 21:33:02.239594 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:02.239567 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:02.240088 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:02.239697 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:02.240088 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:02.239901 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:02.240088 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:02.239998 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:02.847135 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:02.846194 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9gck\" (UniqueName: \"kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck\") pod \"network-check-target-wlk2h\" (UID: \"1dc458d1-00fb-427f-b71b-612cab976ce1\") " pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:02.847135 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:02.846246 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs\") pod \"network-metrics-daemon-8bgg4\" (UID: \"57804798-9f60-42ad-82f9-8ff4065af152\") " pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:02.847135 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:02.846277 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret\") pod \"global-pull-secret-syncer-jbjl2\" (UID: \"1eac0252-00ff-4edd-a198-158b8a99ba18\") " pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:02.847135 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:02.846419 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:33:02.847135 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:02.846478 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret podName:1eac0252-00ff-4edd-a198-158b8a99ba18 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:06.846460114 +0000 UTC m=+9.144345030 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret") pod "global-pull-secret-syncer-jbjl2" (UID: "1eac0252-00ff-4edd-a198-158b8a99ba18") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:33:02.847135 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:02.846890 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:33:02.847135 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:02.846912 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:33:02.847135 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:02.846924 2580 projected.go:196] Error preparing data for projected volume kube-api-access-j9gck for pod openshift-network-diagnostics/network-check-target-wlk2h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:02.847135 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:02.846981 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck podName:1dc458d1-00fb-427f-b71b-612cab976ce1 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:06.846954223 +0000 UTC m=+9.144839138 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9gck" (UniqueName: "kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck") pod "network-check-target-wlk2h" (UID: "1dc458d1-00fb-427f-b71b-612cab976ce1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:02.847135 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:02.847049 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:02.847135 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:02.847080 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs podName:57804798-9f60-42ad-82f9-8ff4065af152 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:06.847068897 +0000 UTC m=+9.144953813 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs") pod "network-metrics-daemon-8bgg4" (UID: "57804798-9f60-42ad-82f9-8ff4065af152") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:03.236825 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:03.236732 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:03.236992 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:03.236879 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:04.246128 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:04.246061 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:04.246128 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:04.246077 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:04.246603 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:04.246265 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:04.246752 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:04.246729 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:05.237563 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:05.237121 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:05.237563 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:05.237234 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:06.237156 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:06.237077 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:06.237156 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:06.237098 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:06.237626 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:06.237227 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:06.237626 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:06.237351 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:06.879272 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:06.879229 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9gck\" (UniqueName: \"kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck\") pod \"network-check-target-wlk2h\" (UID: \"1dc458d1-00fb-427f-b71b-612cab976ce1\") " pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:06.879455 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:06.879344 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs\") pod \"network-metrics-daemon-8bgg4\" (UID: \"57804798-9f60-42ad-82f9-8ff4065af152\") " pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:06.879455 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:06.879386 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret\") pod \"global-pull-secret-syncer-jbjl2\" (UID: \"1eac0252-00ff-4edd-a198-158b8a99ba18\") " pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:06.879455 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:06.879404 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:33:06.879455 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:06.879430 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:33:06.879455 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:06.879445 2580 projected.go:196] Error preparing data for projected volume kube-api-access-j9gck for pod openshift-network-diagnostics/network-check-target-wlk2h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:06.879639 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:06.879495 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck podName:1dc458d1-00fb-427f-b71b-612cab976ce1 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:14.879482295 +0000 UTC m=+17.177367207 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9gck" (UniqueName: "kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck") pod "network-check-target-wlk2h" (UID: "1dc458d1-00fb-427f-b71b-612cab976ce1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:06.879639 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:06.879517 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:06.879639 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:06.879562 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:33:06.879639 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:06.879612 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs podName:57804798-9f60-42ad-82f9-8ff4065af152 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:14.879561002 +0000 UTC m=+17.177446125 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs") pod "network-metrics-daemon-8bgg4" (UID: "57804798-9f60-42ad-82f9-8ff4065af152") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:06.879639 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:06.879632 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret podName:1eac0252-00ff-4edd-a198-158b8a99ba18 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:14.879622687 +0000 UTC m=+17.177507605 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret") pod "global-pull-secret-syncer-jbjl2" (UID: "1eac0252-00ff-4edd-a198-158b8a99ba18") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:33:07.237374 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:07.236890 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:07.237374 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:07.237012 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:08.237738 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:08.237706 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:08.238194 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:08.237850 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:08.238194 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:08.237875 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:08.238194 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:08.237997 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:09.237953 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:09.237427 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:09.237953 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:09.237581 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:10.237979 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:10.237204 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:10.237979 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:10.237336 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:10.237979 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:10.237832 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:10.237979 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:10.237933 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:11.236648 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:11.236627 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:11.236742 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:11.236723 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:12.237352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:12.237319 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:12.237352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:12.237339 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:12.237892 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:12.237451 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:12.237892 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:12.237561 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:13.237272 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:13.237240 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:13.237456 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:13.237360 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:14.237016 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:14.236981 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:14.237200 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:14.237106 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:14.237200 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:14.237167 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:14.237314 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:14.237292 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:14.939558 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:14.939525 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9gck\" (UniqueName: \"kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck\") pod \"network-check-target-wlk2h\" (UID: \"1dc458d1-00fb-427f-b71b-612cab976ce1\") " pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:14.939558 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:14.939572 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs\") pod \"network-metrics-daemon-8bgg4\" (UID: \"57804798-9f60-42ad-82f9-8ff4065af152\") " pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:14.940140 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:14.939687 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:14.940140 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:14.939718 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:33:14.940140 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:14.939737 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs podName:57804798-9f60-42ad-82f9-8ff4065af152 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:30.939721798 +0000 UTC m=+33.237606710 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs") pod "network-metrics-daemon-8bgg4" (UID: "57804798-9f60-42ad-82f9-8ff4065af152") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:14.940140 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:14.939742 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:33:14.940140 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:14.939756 2580 projected.go:196] Error preparing data for projected volume kube-api-access-j9gck for pod openshift-network-diagnostics/network-check-target-wlk2h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:14.940140 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:14.939805 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck podName:1dc458d1-00fb-427f-b71b-612cab976ce1 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:30.939789382 +0000 UTC m=+33.237674302 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9gck" (UniqueName: "kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck") pod "network-check-target-wlk2h" (UID: "1dc458d1-00fb-427f-b71b-612cab976ce1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:14.940140 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:14.939861 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret\") pod \"global-pull-secret-syncer-jbjl2\" (UID: \"1eac0252-00ff-4edd-a198-158b8a99ba18\") " pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:14.940140 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:14.939991 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:33:14.940140 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:14.940028 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret podName:1eac0252-00ff-4edd-a198-158b8a99ba18 nodeName:}" failed. No retries permitted until 2026-07-08 21:33:30.940017336 +0000 UTC m=+33.237902269 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret") pod "global-pull-secret-syncer-jbjl2" (UID: "1eac0252-00ff-4edd-a198-158b8a99ba18") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:33:15.236569 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:15.236497 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:15.236712 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:15.236619 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:16.236797 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:16.236744 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:16.237219 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:16.236761 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:16.237219 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:16.236928 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:16.237219 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:16.236971 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:17.236763 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:17.236734 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:17.236998 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:17.236845 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:18.237273 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:18.237236 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:18.237653 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:18.237315 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:18.237653 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:18.237337 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:18.237653 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:18.237407 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:19.237007 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:19.236976 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:19.237152 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:19.237074 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:20.236604 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.236439 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:20.237256 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.236439 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:20.237256 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:20.236702 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:20.237256 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:20.236768 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:20.438849 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.438659 2580 generic.go:356] "Generic (PLEG): container finished" podID="b2bbc34fbe55f79f66e34f21916774a0" containerID="49352035cf7814047274a71c732934f9d947fb664f61be257135c3ec0d86d8dd" exitCode=0 Jul 08 21:33:20.438986 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.438739 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal" event={"ID":"b2bbc34fbe55f79f66e34f21916774a0","Type":"ContainerDied","Data":"49352035cf7814047274a71c732934f9d947fb664f61be257135c3ec0d86d8dd"} Jul 08 21:33:20.440496 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.440471 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-229.ec2.internal" event={"ID":"4a1b4940af681a5e34bbf5436c61df86","Type":"ContainerStarted","Data":"61ab23a6bc4e06dd35c9ec4b7417fda70e4921870cc92a5fb912e2a9e10319c2"} Jul 08 21:33:20.442395 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.442362 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gdk2v" event={"ID":"b068ba38-cb80-4fa3-b474-294c4d2876a9","Type":"ContainerStarted","Data":"032bc9336cbed9e06960accf558900f9d3181a2bba3e1e0ef14dc84c4b887e41"} Jul 08 21:33:20.445724 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.445701 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" event={"ID":"159d4a44-ebf1-4d5e-913f-321bb9b767f8","Type":"ContainerStarted","Data":"a524976358b5500b13232dcda5345c34b971755ba341dbd4ab12038f5204134a"} Jul 08 21:33:20.445863 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.445763 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" event={"ID":"159d4a44-ebf1-4d5e-913f-321bb9b767f8","Type":"ContainerStarted","Data":"740dbf677860b5b7436b4109d26cb560eb27133db6ca8d081a5be5a7b294279a"} Jul 08 21:33:20.445863 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.445802 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" event={"ID":"159d4a44-ebf1-4d5e-913f-321bb9b767f8","Type":"ContainerStarted","Data":"0352c70085ade6456220dfbee4a87a879870f25827cafc69f5c3c433fcece88f"} Jul 08 21:33:20.445863 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.445861 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" event={"ID":"159d4a44-ebf1-4d5e-913f-321bb9b767f8","Type":"ContainerStarted","Data":"7d6ca3bded0ffc7f3ad2209587f56d8067df3b729b282f2240ef74d2efee76da"} Jul 08 21:33:20.446003 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.445897 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" event={"ID":"159d4a44-ebf1-4d5e-913f-321bb9b767f8","Type":"ContainerStarted","Data":"906a05fb4367e4d20405485fffb848e3b3d6b8c86fb6e68389b74944b14924c3"} Jul 08 21:33:20.446003 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.445934 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" event={"ID":"159d4a44-ebf1-4d5e-913f-321bb9b767f8","Type":"ContainerStarted","Data":"6373b2e994cf4595537ed35a125818d9110e7ccafd1eab1459aefc0c535fdb65"} Jul 08 21:33:20.447321 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.447296 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vlzqf" event={"ID":"dcc5996d-a926-4802-853f-c4991d8ecbef","Type":"ContainerStarted","Data":"9a7a072cb4c00fb5ac3f8e50103f1cadf7c926298d73f8833618edc0c2bf8b9f"} Jul 08 21:33:20.449056 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.449031 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" event={"ID":"a17f0dcb-0507-43d8-8a2d-b313d8fad47f","Type":"ContainerStarted","Data":"36ddf18e5b957cc63d17a28b0196237bddb865546062fc941676926eb62640a9"} Jul 08 21:33:20.450358 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.450328 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bcglp" event={"ID":"990847bf-9c74-4725-84b0-fb3c2f05a5c2","Type":"ContainerStarted","Data":"a4e9f1b89c1467356fe82025e933d4901872021d8dab7fdc88d59499749dee6e"} Jul 08 21:33:20.451851 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.451827 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9p7zn" event={"ID":"09c797d3-78e5-472e-b000-c66f6075ba77","Type":"ContainerStarted","Data":"947b90e135a0229a0f7485fe0029b797d9ca1ca205f79b597221d5b7620fb900"} Jul 08 21:33:20.453524 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.453497 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vpcft" event={"ID":"e4803b19-4882-41f4-98a7-129d0493c630","Type":"ContainerStarted","Data":"6f65f349a633ba26e23d64f2711ca8f3360846165c506d59e88bd8de10adde4e"} Jul 08 21:33:20.457083 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.457058 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" event={"ID":"736f94ce-c503-42ea-876b-b23b90389e46","Type":"ContainerStarted","Data":"eed8fe7d9cea299a82da366cb42e2127491cb3b252d622bc0b781f4e10a7e700"} Jul 08 21:33:20.468920 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.468884 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vpcft" podStartSLOduration=2.369219646 podStartE2EDuration="22.46887196s" totalImagesPullingTime="20.099652314s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:20.468635346 +0000 UTC m=+22.766520281" watchObservedRunningTime="2026-07-08 21:33:20.46887196 +0000 UTC m=+22.766756892" Jul 08 21:33:20.484725 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.484674 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bcglp" podStartSLOduration=10.845013426 podStartE2EDuration="22.484658262s" totalImagesPullingTime="11.639644836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:20.484369728 +0000 UTC m=+22.782254664" watchObservedRunningTime="2026-07-08 21:33:20.484658262 +0000 UTC m=+22.782543198" Jul 08 21:33:20.518559 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.518523 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-gdk2v" podStartSLOduration=6.574442803 podStartE2EDuration="22.518509485s" totalImagesPullingTime="15.944066682s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:20.501484486 +0000 UTC m=+22.799369423" watchObservedRunningTime="2026-07-08 21:33:20.518509485 +0000 UTC m=+22.816394419" Jul 08 21:33:20.518677 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.518658 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-229.ec2.internal" podStartSLOduration=21.518653714 podStartE2EDuration="21.518653714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:20.518202849 +0000 UTC m=+22.816087785" watchObservedRunningTime="2026-07-08 21:33:20.518653714 +0000 UTC m=+22.816538649" Jul 08 21:33:20.565385 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.565342 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vlzqf" podStartSLOduration=2.423678345 podStartE2EDuration="22.565329415s" totalImagesPullingTime="20.14165107s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:20.565236066 +0000 UTC m=+22.863121000" watchObservedRunningTime="2026-07-08 21:33:20.565329415 +0000 UTC m=+22.863214349" Jul 08 21:33:20.586071 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.585996 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-h7h7c" podStartSLOduration=2.446248625 podStartE2EDuration="22.585982934s" totalImagesPullingTime="20.139734309s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:20.58540012 +0000 UTC m=+22.883285055" watchObservedRunningTime="2026-07-08 21:33:20.585982934 +0000 UTC m=+22.883867868" Jul 08 21:33:20.735797 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.735762 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:20.821272 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:20.821245 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:21.025075 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:21.024993 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:21.118760 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:21.118729 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:21.236889 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:21.236859 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:21.237264 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:21.236980 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:21.460751 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:21.460680 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nt658" event={"ID":"7c115713-6b07-43e3-bcba-349ca1c338ef","Type":"ContainerStarted","Data":"9d8696b19158961ef91c3219f0509465c7ae255f5cddd12cb10990efe2dce821"} Jul 08 21:33:21.461986 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:21.461955 2580 generic.go:356] "Generic (PLEG): container finished" podID="09c797d3-78e5-472e-b000-c66f6075ba77" containerID="947b90e135a0229a0f7485fe0029b797d9ca1ca205f79b597221d5b7620fb900" exitCode=0 Jul 08 21:33:21.462104 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:21.462045 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9p7zn" event={"ID":"09c797d3-78e5-472e-b000-c66f6075ba77","Type":"ContainerDied","Data":"947b90e135a0229a0f7485fe0029b797d9ca1ca205f79b597221d5b7620fb900"} Jul 08 21:33:21.463803 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:21.463750 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal" event={"ID":"b2bbc34fbe55f79f66e34f21916774a0","Type":"ContainerStarted","Data":"21141b7cbeb3c5f19f623b166ae2aec5b21aaf18149d39a24395f67ace451ec6"} Jul 08 21:33:21.471306 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:21.471288 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:21.505898 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:21.505860 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nt658" podStartSLOduration=3.466111461 podStartE2EDuration="23.50584567s" totalImagesPullingTime="20.039734209s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:21.482649536 +0000 UTC m=+23.780534470" watchObservedRunningTime="2026-07-08 21:33:21.50584567 +0000 UTC m=+23.803730605" Jul 08 21:33:21.539107 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:21.539065 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-229.ec2.internal" podStartSLOduration=22.539050562 podStartE2EDuration="22.539050562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:21.505906964 +0000 UTC m=+23.803791898" watchObservedRunningTime="2026-07-08 21:33:21.539050562 +0000 UTC m=+23.836935496" Jul 08 21:33:21.574097 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:21.574070 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:21.674766 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:21.674740 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:21.772971 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:21.772950 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:21.797639 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:21.797614 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 21:33:21.914166 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:21.914133 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:21.988439 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:21.988371 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:22.182632 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:22.182492 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T21:33:21.797629404Z","UUID":"c3a9e2a1-1edb-446d-a489-d8880f1fcdbb","Handler":null,"Name":"","Endpoint":""} Jul 08 21:33:22.184017 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:22.183997 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 21:33:22.184149 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:22.184039 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 21:33:22.206088 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:22.206059 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:22.236622 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:22.236597 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:22.236797 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:22.236739 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:22.236953 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:22.236798 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:22.237289 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:22.237250 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:22.307039 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:22.306970 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:22.472430 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:22.471989 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" event={"ID":"a17f0dcb-0507-43d8-8a2d-b313d8fad47f","Type":"ContainerStarted","Data":"4ec74368ddfa570d8e115330c2c08a994a66b7324797fcdae1a44722b6f9467a"} Jul 08 21:33:22.658905 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:22.658831 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:22.732526 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:22.732489 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:22.972534 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:22.972443 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:23.058829 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:23.058788 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gdk2v_b068ba38-cb80-4fa3-b474-294c4d2876a9/dns-node-resolver/0.log" Jul 08 21:33:23.083083 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:23.083051 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:23.182372 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:23.182347 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:23.236667 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:23.236601 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:23.236777 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:23.236705 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:23.476929 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:23.476891 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" event={"ID":"159d4a44-ebf1-4d5e-913f-321bb9b767f8","Type":"ContainerStarted","Data":"4f9383515d3abd2e1defd1f8f7b5fbc1a18517e38f0aad779660b2af1be73cac"} Jul 08 21:33:23.478709 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:23.478678 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" event={"ID":"a17f0dcb-0507-43d8-8a2d-b313d8fad47f","Type":"ContainerStarted","Data":"043e9701d721becfc456c870efc94914685a8e0b855639664a5ae3e371ee10ed"} Jul 08 21:33:23.499361 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:23.499296 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mp2ht" podStartSLOduration=2.874997605 podStartE2EDuration="25.49928534s" totalImagesPullingTime="22.624287735s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:23.499011657 +0000 UTC m=+25.796896594" watchObservedRunningTime="2026-07-08 21:33:23.49928534 +0000 UTC m=+25.797170275" Jul 08 21:33:23.704277 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:23.704231 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:23.802961 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:23.802866 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:23.911282 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:23.911254 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bcglp" Jul 08 21:33:23.911769 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:23.911751 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bcglp" Jul 08 21:33:24.237054 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:24.236972 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:24.237204 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:24.236977 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:24.237204 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:24.237115 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:24.237204 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:24.237182 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:24.247308 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:24.247284 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vpcft_e4803b19-4882-41f4-98a7-129d0493c630/node-ca/0.log" Jul 08 21:33:24.481178 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:24.481147 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bcglp" Jul 08 21:33:24.481688 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:24.481668 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bcglp" Jul 08 21:33:24.557391 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:24.557360 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:24.614782 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:24.614751 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:24.774741 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:24.774709 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:24.868576 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:24.868498 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:25.236836 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:25.236750 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:25.236994 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:25.236891 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:25.486645 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:25.486466 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:26.236726 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:26.236657 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:26.236878 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:26.236662 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:26.236878 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:26.236754 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:26.236961 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:26.236889 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:26.488715 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:26.488657 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" event={"ID":"159d4a44-ebf1-4d5e-913f-321bb9b767f8","Type":"ContainerStarted","Data":"41ab4663923a9f1dbfdfc713802cda992f9a351525ddf5e7aaefed5954f6bec2"} Jul 08 21:33:26.489430 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:26.488904 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:33:26.490600 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:26.490577 2580 generic.go:356] "Generic (PLEG): container finished" podID="09c797d3-78e5-472e-b000-c66f6075ba77" containerID="5a87d521e7e0e0449d1e7bb14a04869b456d56fa82b8a8fbed1c64d178a4935f" exitCode=0 Jul 08 21:33:26.490732 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:26.490666 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9p7zn" event={"ID":"09c797d3-78e5-472e-b000-c66f6075ba77","Type":"ContainerDied","Data":"5a87d521e7e0e0449d1e7bb14a04869b456d56fa82b8a8fbed1c64d178a4935f"} Jul 08 21:33:26.504701 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:26.504674 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:33:26.536904 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:26.536861 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" podStartSLOduration=8.330276912 podStartE2EDuration="28.536848677s" totalImagesPullingTime="20.206571765s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:26.535635637 +0000 UTC m=+28.833520572" watchObservedRunningTime="2026-07-08 21:33:26.536848677 +0000 UTC m=+28.834733611" Jul 08 21:33:26.818533 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:26.818287 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:27.094777 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:27.094700 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:27.202112 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:27.202074 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:27.236974 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:27.236945 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:27.237086 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:27.237061 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:27.319889 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:27.319842 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:27.493622 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:27.493547 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:33:27.493622 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:27.493609 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:33:27.508246 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:27.508224 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:33:27.517197 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:27.517177 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:28.238230 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:28.238151 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:28.238366 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:28.238236 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:28.238366 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:28.238263 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:28.238366 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:28.238319 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:28.497528 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:28.497452 2580 generic.go:356] "Generic (PLEG): container finished" podID="09c797d3-78e5-472e-b000-c66f6075ba77" containerID="248147a15a42ad8f39189cb168883e47c7d40ae3262c7f7d4acd6e4b9e8ac2d4" exitCode=0 Jul 08 21:33:28.497883 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:28.497541 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9p7zn" event={"ID":"09c797d3-78e5-472e-b000-c66f6075ba77","Type":"ContainerDied","Data":"248147a15a42ad8f39189cb168883e47c7d40ae3262c7f7d4acd6e4b9e8ac2d4"} Jul 08 21:33:28.510098 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:28.510075 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:28.734253 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:28.734209 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:29.130397 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:29.130369 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:29.236821 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:29.236777 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:29.236949 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:29.236887 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:29.333545 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:29.333514 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:29.445024 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:29.444957 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:29.571457 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:29.571421 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:29.729472 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:29.729445 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:30.237509 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:30.237477 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:30.237657 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:30.237511 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:30.237657 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:30.237619 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:30.237748 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:30.237729 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:30.502899 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:30.502802 2580 generic.go:356] "Generic (PLEG): container finished" podID="09c797d3-78e5-472e-b000-c66f6075ba77" containerID="76ae4e3b9ed8ee6f172a68c3e187cf4680f94e236a66ac998b6ba255db76ebcb" exitCode=0 Jul 08 21:33:30.502899 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:30.502876 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9p7zn" event={"ID":"09c797d3-78e5-472e-b000-c66f6075ba77","Type":"ContainerDied","Data":"76ae4e3b9ed8ee6f172a68c3e187cf4680f94e236a66ac998b6ba255db76ebcb"} Jul 08 21:33:30.555697 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:30.555670 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:30.655513 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:30.655484 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:30.717626 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:30.717594 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:30.962673 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:30.962601 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9gck\" (UniqueName: \"kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck\") pod \"network-check-target-wlk2h\" (UID: \"1dc458d1-00fb-427f-b71b-612cab976ce1\") " pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:30.962673 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:30.962638 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs\") pod \"network-metrics-daemon-8bgg4\" (UID: \"57804798-9f60-42ad-82f9-8ff4065af152\") " pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:30.962673 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:30.962661 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret\") pod \"global-pull-secret-syncer-jbjl2\" (UID: \"1eac0252-00ff-4edd-a198-158b8a99ba18\") " pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:30.962914 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:30.962761 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:30.962914 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:30.962773 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 21:33:30.962914 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:30.962778 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 21:33:30.962914 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:30.962796 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 21:33:30.962914 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:30.962826 2580 projected.go:196] Error preparing data for projected volume kube-api-access-j9gck for pod openshift-network-diagnostics/network-check-target-wlk2h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:30.962914 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:30.962832 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs podName:57804798-9f60-42ad-82f9-8ff4065af152 nodeName:}" failed. No retries permitted until 2026-07-08 21:34:02.96280455 +0000 UTC m=+65.260689466 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs") pod "network-metrics-daemon-8bgg4" (UID: "57804798-9f60-42ad-82f9-8ff4065af152") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 21:33:30.962914 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:30.962848 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret podName:1eac0252-00ff-4edd-a198-158b8a99ba18 nodeName:}" failed. No retries permitted until 2026-07-08 21:34:02.962841234 +0000 UTC m=+65.260726150 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret") pod "global-pull-secret-syncer-jbjl2" (UID: "1eac0252-00ff-4edd-a198-158b8a99ba18") : object "kube-system"/"original-pull-secret" not registered Jul 08 21:33:30.962914 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:30.962866 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck podName:1dc458d1-00fb-427f-b71b-612cab976ce1 nodeName:}" failed. No retries permitted until 2026-07-08 21:34:02.962855594 +0000 UTC m=+65.260740509 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9gck" (UniqueName: "kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck") pod "network-check-target-wlk2h" (UID: "1dc458d1-00fb-427f-b71b-612cab976ce1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 21:33:31.115797 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:31.115756 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:31.236672 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:31.236606 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:31.236803 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:31.236706 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:31.314189 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:31.314161 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:31.388271 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:31.388225 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:31.517414 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:31.517329 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:32.236591 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:32.236555 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:32.236591 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:32.236595 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:32.237122 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:32.236682 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:32.237122 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:32.236761 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:32.716212 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:32.716122 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:32.930172 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:32.930141 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:33.056664 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:33.056577 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:33.155968 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:33.155938 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:33.236958 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:33.236930 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:33.237349 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:33.237039 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:33.313883 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:33.313795 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:33.522982 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:33.522950 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:34.236968 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:34.236930 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:34.237357 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:34.236981 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:34.237357 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:34.237086 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:34.237357 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:34.237212 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:34.537283 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:34.537198 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:34.711830 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:34.711783 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:35.102019 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:35.101991 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:35.205151 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:35.205101 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:35.237140 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:35.237110 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:35.237548 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:35.237225 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:35.310433 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:35.310402 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:35.737047 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:35.736974 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:36.237125 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:36.237095 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:36.237292 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:36.237198 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:36.237292 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:36.237274 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:36.237589 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:36.237356 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:36.292486 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:36.292450 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:36.401715 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:36.401681 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:36.498160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:36.498092 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:36.520112 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:36.520088 2580 generic.go:356] "Generic (PLEG): container finished" podID="09c797d3-78e5-472e-b000-c66f6075ba77" containerID="57972d7913fe4c2d01e7df68020be55cb8b4e36c265c925709c9ab7950b30c03" exitCode=0 Jul 08 21:33:36.520248 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:36.520134 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9p7zn" event={"ID":"09c797d3-78e5-472e-b000-c66f6075ba77","Type":"ContainerDied","Data":"57972d7913fe4c2d01e7df68020be55cb8b4e36c265c925709c9ab7950b30c03"} Jul 08 21:33:36.590011 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:36.589990 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:36.918878 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:36.918845 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:37.017494 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:37.017468 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:37.138397 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:37.138370 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:37.236881 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:37.236798 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:37.237007 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:37.236917 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:37.239755 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:37.239735 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:37.417222 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:37.417195 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:37.511947 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:37.511883 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:37.525157 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:37.525133 2580 generic.go:356] "Generic (PLEG): container finished" podID="09c797d3-78e5-472e-b000-c66f6075ba77" containerID="b6997734fbe4edb15aa89e0b6bd7d9aa281c0035f79a42803fecca9214049192" exitCode=0 Jul 08 21:33:37.525290 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:37.525190 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9p7zn" event={"ID":"09c797d3-78e5-472e-b000-c66f6075ba77","Type":"ContainerDied","Data":"b6997734fbe4edb15aa89e0b6bd7d9aa281c0035f79a42803fecca9214049192"} Jul 08 21:33:37.608712 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:37.608682 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:37.975825 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:37.975791 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:38.104017 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:38.103971 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:38.204704 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:38.204675 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:38.237877 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:38.237798 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:38.237877 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:38.237859 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:38.238027 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:38.237936 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:38.238082 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:38.238063 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:38.379300 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:38.379267 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:38.531099 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:38.531067 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9p7zn" event={"ID":"09c797d3-78e5-472e-b000-c66f6075ba77","Type":"ContainerStarted","Data":"5a01224b4e9208c98d628209d3b8e1103c64f1490607090ce266f43a37a620d8"} Jul 08 21:33:38.559312 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:38.559266 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9p7zn" podStartSLOduration=12.490126644 podStartE2EDuration="40.559254109s" totalImagesPullingTime="28.069127465s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:33:38.559103599 +0000 UTC m=+40.856988534" watchObservedRunningTime="2026-07-08 21:33:38.559254109 +0000 UTC m=+40.857139044" Jul 08 21:33:38.665002 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:38.664975 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:38.741587 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:38.741558 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:38.995356 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:38.995287 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:39.097429 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:39.097402 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:39.236483 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:39.236456 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:39.236646 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:39.236547 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:40.237282 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:40.237249 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:40.237607 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:40.237381 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:40.237607 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:40.237431 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:40.237607 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:40.237506 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:41.237316 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:41.237283 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:41.237687 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:41.237382 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:42.236712 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:42.236669 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:42.236931 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:42.236778 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:42.236931 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:42.236833 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:42.236931 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:42.236921 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:43.236951 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:43.236919 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:43.237383 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:43.237020 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:44.236666 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:44.236631 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:44.236841 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:44.236772 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:44.236895 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:44.236842 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:44.236977 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:44.236924 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:45.236480 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:45.236447 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:45.236670 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:45.236557 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:45.988671 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:45.988633 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:46.087673 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:46.087638 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:46.239515 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:46.239447 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:46.239648 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:46.239533 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:46.239648 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:46.239588 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:46.239718 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:46.239681 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:47.236727 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:47.236700 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:47.237112 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:47.236787 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:47.995181 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:47.995152 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:48.050570 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:48.050539 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:48.237170 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:48.237135 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:48.237623 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:48.237244 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:48.237623 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:48.237290 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:48.237623 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:48.237341 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:48.875761 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:48.875731 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:49.033330 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:49.033299 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:49.200853 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:49.200758 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:49.237000 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:49.236961 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:49.237161 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:49.237122 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:49.311284 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:49.311249 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:50.237003 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:50.236834 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:50.237132 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:50.236883 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:50.237132 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:50.237095 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:50.237212 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:50.237151 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:50.356750 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:50.356700 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8bgg4"] Jul 08 21:33:50.357294 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:50.357271 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wlk2h"] Jul 08 21:33:50.357421 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:50.357380 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:50.357506 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:50.357485 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:50.369103 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:50.369037 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jbjl2"] Jul 08 21:33:50.489076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:50.489043 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:50.558375 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:50.558348 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:50.558517 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:50.558349 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:50.558517 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:50.558467 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:50.558598 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:50.558519 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:50.584770 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:50.584742 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:50.657785 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:50.657717 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:50.885655 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:50.885630 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:51.172140 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:51.172111 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:51.303161 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:51.303129 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:52.236409 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:52.236380 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:52.236961 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:52.236410 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:52.236961 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:52.236499 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:52.236961 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:52.236552 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:52.236961 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:52.236666 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:52.236961 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:52.236700 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:53.626089 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:53.626056 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:53.680084 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:53.680048 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:53.954438 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:53.954365 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:54.237479 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.237402 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:54.237479 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.237431 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:54.237479 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.237482 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:54.237709 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:54.237490 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlk2h" podUID="1dc458d1-00fb-427f-b71b-612cab976ce1" Jul 08 21:33:54.237709 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:54.237573 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jbjl2" podUID="1eac0252-00ff-4edd-a198-158b8a99ba18" Jul 08 21:33:54.237709 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:33:54.237653 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bgg4" podUID="57804798-9f60-42ad-82f9-8ff4065af152" Jul 08 21:33:54.532484 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.532462 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-229.ec2.internal" event="NodeReady" Jul 08 21:33:54.532640 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.532595 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 21:33:54.573847 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.573801 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wdmtl"] Jul 08 21:33:54.575545 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.575529 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wdmtl" Jul 08 21:33:54.577023 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.577004 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dpk92"] Jul 08 21:33:54.578199 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.578177 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 21:33:54.578294 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.578177 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 21:33:54.578294 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.578201 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 21:33:54.578403 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.578323 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kngpn\"" Jul 08 21:33:54.578595 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.578578 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dpk92" Jul 08 21:33:54.581090 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.581073 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 21:33:54.581396 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.581375 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-65cm9\"" Jul 08 21:33:54.581489 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.581379 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 21:33:54.581755 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.581733 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 21:33:54.581755 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.581747 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 21:33:54.586671 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.586651 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wdmtl"] Jul 08 21:33:54.592430 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.592411 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dpk92"] Jul 08 21:33:54.596481 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.596457 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cngfd"] Jul 08 21:33:54.598005 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.597992 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cngfd" Jul 08 21:33:54.600433 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.600398 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 21:33:54.600524 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.600445 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 21:33:54.600585 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.600567 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wfpxn\"" Jul 08 21:33:54.608991 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.608967 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cngfd"] Jul 08 21:33:54.646921 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.646892 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nstdb\" (UniqueName: \"kubernetes.io/projected/bc6e0d63-2587-450e-9001-9e1a13874ad6-kube-api-access-nstdb\") pod \"dns-default-cngfd\" (UID: \"bc6e0d63-2587-450e-9001-9e1a13874ad6\") " pod="openshift-dns/dns-default-cngfd" Jul 08 21:33:54.647214 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.646965 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4075449-512f-49ac-9cdf-65c97d14f19a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dpk92\" (UID: \"c4075449-512f-49ac-9cdf-65c97d14f19a\") " pod="openshift-insights/insights-runtime-extractor-dpk92" Jul 08 21:33:54.647214 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.647011 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c4075449-512f-49ac-9cdf-65c97d14f19a-crio-socket\") pod \"insights-runtime-extractor-dpk92\" (UID: \"c4075449-512f-49ac-9cdf-65c97d14f19a\") " pod="openshift-insights/insights-runtime-extractor-dpk92" Jul 08 21:33:54.647214 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.647031 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjlsr\" (UniqueName: \"kubernetes.io/projected/c4075449-512f-49ac-9cdf-65c97d14f19a-kube-api-access-wjlsr\") pod \"insights-runtime-extractor-dpk92\" (UID: \"c4075449-512f-49ac-9cdf-65c97d14f19a\") " pod="openshift-insights/insights-runtime-extractor-dpk92" Jul 08 21:33:54.647214 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.647048 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bc6e0d63-2587-450e-9001-9e1a13874ad6-config-volume\") pod \"dns-default-cngfd\" (UID: \"bc6e0d63-2587-450e-9001-9e1a13874ad6\") " pod="openshift-dns/dns-default-cngfd" Jul 08 21:33:54.647214 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.647062 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc6e0d63-2587-450e-9001-9e1a13874ad6-metrics-tls\") pod \"dns-default-cngfd\" (UID: \"bc6e0d63-2587-450e-9001-9e1a13874ad6\") " pod="openshift-dns/dns-default-cngfd" Jul 08 21:33:54.647214 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.647077 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc6e0d63-2587-450e-9001-9e1a13874ad6-tmp-dir\") pod \"dns-default-cngfd\" (UID: \"bc6e0d63-2587-450e-9001-9e1a13874ad6\") " pod="openshift-dns/dns-default-cngfd" Jul 08 21:33:54.647214 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.647114 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/75941359-0abf-4810-87c8-76ea8cc35f65-cert\") pod \"ingress-canary-wdmtl\" (UID: \"75941359-0abf-4810-87c8-76ea8cc35f65\") " pod="openshift-ingress-canary/ingress-canary-wdmtl" Jul 08 21:33:54.647214 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.647144 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4075449-512f-49ac-9cdf-65c97d14f19a-data-volume\") pod \"insights-runtime-extractor-dpk92\" (UID: \"c4075449-512f-49ac-9cdf-65c97d14f19a\") " pod="openshift-insights/insights-runtime-extractor-dpk92" Jul 08 21:33:54.647214 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.647180 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4075449-512f-49ac-9cdf-65c97d14f19a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dpk92\" (UID: \"c4075449-512f-49ac-9cdf-65c97d14f19a\") " pod="openshift-insights/insights-runtime-extractor-dpk92" Jul 08 21:33:54.647214 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.647201 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j5pr\" (UniqueName: \"kubernetes.io/projected/75941359-0abf-4810-87c8-76ea8cc35f65-kube-api-access-6j5pr\") pod \"ingress-canary-wdmtl\" (UID: \"75941359-0abf-4810-87c8-76ea8cc35f65\") " pod="openshift-ingress-canary/ingress-canary-wdmtl" Jul 08 21:33:54.710446 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.710412 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:54.747624 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.747600 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4075449-512f-49ac-9cdf-65c97d14f19a-data-volume\") pod \"insights-runtime-extractor-dpk92\" (UID: \"c4075449-512f-49ac-9cdf-65c97d14f19a\") " pod="openshift-insights/insights-runtime-extractor-dpk92" Jul 08 21:33:54.747710 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.747639 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4075449-512f-49ac-9cdf-65c97d14f19a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dpk92\" (UID: \"c4075449-512f-49ac-9cdf-65c97d14f19a\") " pod="openshift-insights/insights-runtime-extractor-dpk92" Jul 08 21:33:54.747710 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.747660 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j5pr\" (UniqueName: \"kubernetes.io/projected/75941359-0abf-4810-87c8-76ea8cc35f65-kube-api-access-6j5pr\") pod \"ingress-canary-wdmtl\" (UID: \"75941359-0abf-4810-87c8-76ea8cc35f65\") " pod="openshift-ingress-canary/ingress-canary-wdmtl" Jul 08 21:33:54.747856 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.747781 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nstdb\" (UniqueName: \"kubernetes.io/projected/bc6e0d63-2587-450e-9001-9e1a13874ad6-kube-api-access-nstdb\") pod \"dns-default-cngfd\" (UID: \"bc6e0d63-2587-450e-9001-9e1a13874ad6\") " pod="openshift-dns/dns-default-cngfd" Jul 08 21:33:54.747923 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.747878 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4075449-512f-49ac-9cdf-65c97d14f19a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dpk92\" (UID: \"c4075449-512f-49ac-9cdf-65c97d14f19a\") " pod="openshift-insights/insights-runtime-extractor-dpk92" Jul 08 21:33:54.747923 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.747910 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c4075449-512f-49ac-9cdf-65c97d14f19a-crio-socket\") pod \"insights-runtime-extractor-dpk92\" (UID: \"c4075449-512f-49ac-9cdf-65c97d14f19a\") " pod="openshift-insights/insights-runtime-extractor-dpk92" Jul 08 21:33:54.748021 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.747930 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjlsr\" (UniqueName: \"kubernetes.io/projected/c4075449-512f-49ac-9cdf-65c97d14f19a-kube-api-access-wjlsr\") pod \"insights-runtime-extractor-dpk92\" (UID: \"c4075449-512f-49ac-9cdf-65c97d14f19a\") " pod="openshift-insights/insights-runtime-extractor-dpk92" Jul 08 21:33:54.748021 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.747956 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bc6e0d63-2587-450e-9001-9e1a13874ad6-config-volume\") pod \"dns-default-cngfd\" (UID: \"bc6e0d63-2587-450e-9001-9e1a13874ad6\") " pod="openshift-dns/dns-default-cngfd" Jul 08 21:33:54.748021 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.747977 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc6e0d63-2587-450e-9001-9e1a13874ad6-metrics-tls\") pod \"dns-default-cngfd\" (UID: \"bc6e0d63-2587-450e-9001-9e1a13874ad6\") " pod="openshift-dns/dns-default-cngfd" Jul 08 21:33:54.748021 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.748001 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc6e0d63-2587-450e-9001-9e1a13874ad6-tmp-dir\") pod \"dns-default-cngfd\" (UID: \"bc6e0d63-2587-450e-9001-9e1a13874ad6\") " pod="openshift-dns/dns-default-cngfd" Jul 08 21:33:54.748237 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.748021 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/75941359-0abf-4810-87c8-76ea8cc35f65-cert\") pod \"ingress-canary-wdmtl\" (UID: \"75941359-0abf-4810-87c8-76ea8cc35f65\") " pod="openshift-ingress-canary/ingress-canary-wdmtl" Jul 08 21:33:54.748237 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.748112 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c4075449-512f-49ac-9cdf-65c97d14f19a-crio-socket\") pod \"insights-runtime-extractor-dpk92\" (UID: \"c4075449-512f-49ac-9cdf-65c97d14f19a\") " pod="openshift-insights/insights-runtime-extractor-dpk92" Jul 08 21:33:54.748322 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.748254 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4075449-512f-49ac-9cdf-65c97d14f19a-data-volume\") pod \"insights-runtime-extractor-dpk92\" (UID: \"c4075449-512f-49ac-9cdf-65c97d14f19a\") " pod="openshift-insights/insights-runtime-extractor-dpk92" Jul 08 21:33:54.748458 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.748424 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc6e0d63-2587-450e-9001-9e1a13874ad6-tmp-dir\") pod \"dns-default-cngfd\" (UID: \"bc6e0d63-2587-450e-9001-9e1a13874ad6\") " pod="openshift-dns/dns-default-cngfd" Jul 08 21:33:54.748630 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.748612 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bc6e0d63-2587-450e-9001-9e1a13874ad6-config-volume\") pod \"dns-default-cngfd\" (UID: \"bc6e0d63-2587-450e-9001-9e1a13874ad6\") " pod="openshift-dns/dns-default-cngfd" Jul 08 21:33:54.748710 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.748688 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4075449-512f-49ac-9cdf-65c97d14f19a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dpk92\" (UID: \"c4075449-512f-49ac-9cdf-65c97d14f19a\") " pod="openshift-insights/insights-runtime-extractor-dpk92" Jul 08 21:33:54.751715 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.751691 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4075449-512f-49ac-9cdf-65c97d14f19a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dpk92\" (UID: \"c4075449-512f-49ac-9cdf-65c97d14f19a\") " pod="openshift-insights/insights-runtime-extractor-dpk92" Jul 08 21:33:54.751827 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.751788 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc6e0d63-2587-450e-9001-9e1a13874ad6-metrics-tls\") pod \"dns-default-cngfd\" (UID: \"bc6e0d63-2587-450e-9001-9e1a13874ad6\") " pod="openshift-dns/dns-default-cngfd" Jul 08 21:33:54.751953 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.751938 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/75941359-0abf-4810-87c8-76ea8cc35f65-cert\") pod \"ingress-canary-wdmtl\" (UID: \"75941359-0abf-4810-87c8-76ea8cc35f65\") " pod="openshift-ingress-canary/ingress-canary-wdmtl" Jul 08 21:33:54.758766 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.758747 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjlsr\" (UniqueName: \"kubernetes.io/projected/c4075449-512f-49ac-9cdf-65c97d14f19a-kube-api-access-wjlsr\") pod \"insights-runtime-extractor-dpk92\" (UID: \"c4075449-512f-49ac-9cdf-65c97d14f19a\") " pod="openshift-insights/insights-runtime-extractor-dpk92" Jul 08 21:33:54.759160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.759140 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j5pr\" (UniqueName: \"kubernetes.io/projected/75941359-0abf-4810-87c8-76ea8cc35f65-kube-api-access-6j5pr\") pod \"ingress-canary-wdmtl\" (UID: \"75941359-0abf-4810-87c8-76ea8cc35f65\") " pod="openshift-ingress-canary/ingress-canary-wdmtl" Jul 08 21:33:54.759227 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.759156 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nstdb\" (UniqueName: \"kubernetes.io/projected/bc6e0d63-2587-450e-9001-9e1a13874ad6-kube-api-access-nstdb\") pod \"dns-default-cngfd\" (UID: \"bc6e0d63-2587-450e-9001-9e1a13874ad6\") " pod="openshift-dns/dns-default-cngfd" Jul 08 21:33:54.886924 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.886854 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wdmtl" Jul 08 21:33:54.891254 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.891239 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dpk92" Jul 08 21:33:54.905925 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:54.905900 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cngfd" Jul 08 21:33:55.045067 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:55.045035 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dpk92"] Jul 08 21:33:55.050451 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:33:55.050414 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4075449_512f_49ac_9cdf_65c97d14f19a.slice/crio-f48cdfeede25b15c49f15090094c6b04626286c14013862a249209eb16fd7ef4 WatchSource:0}: Error finding container f48cdfeede25b15c49f15090094c6b04626286c14013862a249209eb16fd7ef4: Status 404 returned error can't find the container with id f48cdfeede25b15c49f15090094c6b04626286c14013862a249209eb16fd7ef4 Jul 08 21:33:55.056430 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:55.056405 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wdmtl"] Jul 08 21:33:55.059337 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:33:55.059308 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75941359_0abf_4810_87c8_76ea8cc35f65.slice/crio-602dfc5b2515b60cb3ffb3950a3987bb130eda448916ab7c1fa55122343dfe30 WatchSource:0}: Error finding container 602dfc5b2515b60cb3ffb3950a3987bb130eda448916ab7c1fa55122343dfe30: Status 404 returned error can't find the container with id 602dfc5b2515b60cb3ffb3950a3987bb130eda448916ab7c1fa55122343dfe30 Jul 08 21:33:55.070018 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:55.069990 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cngfd"] Jul 08 21:33:55.074779 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:33:55.074754 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc6e0d63_2587_450e_9001_9e1a13874ad6.slice/crio-c603d5602a75123c6084851285af21baae2352e49ca6ca802eb8767c66b40bda WatchSource:0}: Error finding container c603d5602a75123c6084851285af21baae2352e49ca6ca802eb8767c66b40bda: Status 404 returned error can't find the container with id c603d5602a75123c6084851285af21baae2352e49ca6ca802eb8767c66b40bda Jul 08 21:33:55.572430 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:55.572383 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dpk92" event={"ID":"c4075449-512f-49ac-9cdf-65c97d14f19a","Type":"ContainerStarted","Data":"f48cdfeede25b15c49f15090094c6b04626286c14013862a249209eb16fd7ef4"} Jul 08 21:33:55.574049 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:55.574018 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wdmtl" event={"ID":"75941359-0abf-4810-87c8-76ea8cc35f65","Type":"ContainerStarted","Data":"602dfc5b2515b60cb3ffb3950a3987bb130eda448916ab7c1fa55122343dfe30"} Jul 08 21:33:55.578015 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:55.577943 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cngfd" event={"ID":"bc6e0d63-2587-450e-9001-9e1a13874ad6","Type":"ContainerStarted","Data":"c603d5602a75123c6084851285af21baae2352e49ca6ca802eb8767c66b40bda"} Jul 08 21:33:56.237705 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:56.237599 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:33:56.237705 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:56.237653 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:33:56.237705 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:56.237671 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:33:56.242364 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:56.242340 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4v6kt\"" Jul 08 21:33:56.242586 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:56.242569 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 21:33:56.242778 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:56.242763 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 21:33:56.243456 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:56.242960 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 21:33:56.243456 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:56.243147 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 21:33:56.243456 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:56.243249 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wsbgf\"" Jul 08 21:33:56.422679 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:56.422641 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:56.483738 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:56.483708 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:58.073564 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:58.073532 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:58.145046 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:58.145013 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:59.517078 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:59.517052 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-czfxp" Jul 08 21:33:59.680706 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:59.680665 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:33:59.779385 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:33:59.779335 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:00.695229 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.695194 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:00.729646 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.729612 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f"] Jul 08 21:34:00.731667 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.731648 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f" Jul 08 21:34:00.734934 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.734480 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 21:34:00.734934 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.734659 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 21:34:00.734934 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.734781 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 08 21:34:00.736068 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.735465 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 08 21:34:00.736068 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.735673 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-pdx5m\"" Jul 08 21:34:00.736068 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.735904 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 21:34:00.746581 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.746557 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f"] Jul 08 21:34:00.768735 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.768661 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:00.792541 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.792514 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03d072a8-9433-41a7-89c0-0e7d2acb08e0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-dkc9f\" (UID: \"03d072a8-9433-41a7-89c0-0e7d2acb08e0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f" Jul 08 21:34:00.792653 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.792555 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/03d072a8-9433-41a7-89c0-0e7d2acb08e0-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-dkc9f\" (UID: \"03d072a8-9433-41a7-89c0-0e7d2acb08e0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f" Jul 08 21:34:00.792653 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.792580 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03d072a8-9433-41a7-89c0-0e7d2acb08e0-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-dkc9f\" (UID: \"03d072a8-9433-41a7-89c0-0e7d2acb08e0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f" Jul 08 21:34:00.792653 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.792595 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvgl4\" (UniqueName: \"kubernetes.io/projected/03d072a8-9433-41a7-89c0-0e7d2acb08e0-kube-api-access-cvgl4\") pod \"prometheus-operator-567f7b8b7c-dkc9f\" (UID: \"03d072a8-9433-41a7-89c0-0e7d2acb08e0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f" Jul 08 21:34:00.893937 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.893904 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03d072a8-9433-41a7-89c0-0e7d2acb08e0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-dkc9f\" (UID: \"03d072a8-9433-41a7-89c0-0e7d2acb08e0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f" Jul 08 21:34:00.894090 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.893947 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/03d072a8-9433-41a7-89c0-0e7d2acb08e0-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-dkc9f\" (UID: \"03d072a8-9433-41a7-89c0-0e7d2acb08e0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f" Jul 08 21:34:00.894090 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.893973 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03d072a8-9433-41a7-89c0-0e7d2acb08e0-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-dkc9f\" (UID: \"03d072a8-9433-41a7-89c0-0e7d2acb08e0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f" Jul 08 21:34:00.894090 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.893990 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvgl4\" (UniqueName: \"kubernetes.io/projected/03d072a8-9433-41a7-89c0-0e7d2acb08e0-kube-api-access-cvgl4\") pod \"prometheus-operator-567f7b8b7c-dkc9f\" (UID: \"03d072a8-9433-41a7-89c0-0e7d2acb08e0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f" Jul 08 21:34:00.894090 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:34:00.894059 2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 08 21:34:00.894241 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:34:00.894135 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03d072a8-9433-41a7-89c0-0e7d2acb08e0-prometheus-operator-tls podName:03d072a8-9433-41a7-89c0-0e7d2acb08e0 nodeName:}" failed. No retries permitted until 2026-07-08 21:34:01.394116559 +0000 UTC m=+63.692001472 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/03d072a8-9433-41a7-89c0-0e7d2acb08e0-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-dkc9f" (UID: "03d072a8-9433-41a7-89c0-0e7d2acb08e0") : secret "prometheus-operator-tls" not found Jul 08 21:34:00.894681 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.894663 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03d072a8-9433-41a7-89c0-0e7d2acb08e0-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-dkc9f\" (UID: \"03d072a8-9433-41a7-89c0-0e7d2acb08e0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f" Jul 08 21:34:00.896432 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.896408 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03d072a8-9433-41a7-89c0-0e7d2acb08e0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-dkc9f\" (UID: \"03d072a8-9433-41a7-89c0-0e7d2acb08e0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f" Jul 08 21:34:00.905143 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:00.905120 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvgl4\" (UniqueName: \"kubernetes.io/projected/03d072a8-9433-41a7-89c0-0e7d2acb08e0-kube-api-access-cvgl4\") pod \"prometheus-operator-567f7b8b7c-dkc9f\" (UID: \"03d072a8-9433-41a7-89c0-0e7d2acb08e0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f" Jul 08 21:34:01.046002 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:01.045957 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:01.164391 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:01.164344 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:01.261250 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:01.261224 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:01.328045 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:01.327969 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:01.399770 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:01.399738 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/03d072a8-9433-41a7-89c0-0e7d2acb08e0-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-dkc9f\" (UID: \"03d072a8-9433-41a7-89c0-0e7d2acb08e0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f" Jul 08 21:34:01.402130 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:01.402112 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/03d072a8-9433-41a7-89c0-0e7d2acb08e0-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-dkc9f\" (UID: \"03d072a8-9433-41a7-89c0-0e7d2acb08e0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f" Jul 08 21:34:01.596274 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:01.596200 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dpk92" event={"ID":"c4075449-512f-49ac-9cdf-65c97d14f19a","Type":"ContainerStarted","Data":"2841fde9b8fdc0890db78bb05fc4a54f464e1296aab1f6653be3a16126813917"} Jul 08 21:34:01.596397 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:01.596288 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dpk92" event={"ID":"c4075449-512f-49ac-9cdf-65c97d14f19a","Type":"ContainerStarted","Data":"adf6a4d73a14ac8b78263e8bfc0d7514f19684976472f9974440d8f20a604dbd"} Jul 08 21:34:01.597574 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:01.597552 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wdmtl" event={"ID":"75941359-0abf-4810-87c8-76ea8cc35f65","Type":"ContainerStarted","Data":"2475eb6b24490f565ca29609e6f240412cc062672d05beb3cb4507a06218685e"} Jul 08 21:34:01.599228 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:01.599206 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cngfd" event={"ID":"bc6e0d63-2587-450e-9001-9e1a13874ad6","Type":"ContainerStarted","Data":"2c7efa4f31c9fb651d3e8f67cc8ed2be946f59216e9992f87fa198dcaf92ddc1"} Jul 08 21:34:01.599317 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:01.599242 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cngfd" event={"ID":"bc6e0d63-2587-450e-9001-9e1a13874ad6","Type":"ContainerStarted","Data":"9fc5099e815e71986f8330ea5e908b62c56a8b615388127d247e4d69dc1a2872"} Jul 08 21:34:01.599382 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:01.599368 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cngfd" Jul 08 21:34:01.616237 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:01.616183 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wdmtl" podStartSLOduration=2.14864773 podStartE2EDuration="7.616170445s" totalImagesPullingTime="5.467522715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:33:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:01.614806534 +0000 UTC m=+63.912691479" watchObservedRunningTime="2026-07-08 21:34:01.616170445 +0000 UTC m=+63.914055379" Jul 08 21:34:01.642388 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:01.642365 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f" Jul 08 21:34:01.649358 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:01.649173 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cngfd" podStartSLOduration=2.215907615 podStartE2EDuration="7.649158591s" totalImagesPullingTime="5.433250976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:33:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:01.648262497 +0000 UTC m=+63.946147424" watchObservedRunningTime="2026-07-08 21:34:01.649158591 +0000 UTC m=+63.947043527" Jul 08 21:34:01.774493 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:01.774462 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f"] Jul 08 21:34:01.777485 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:34:01.777457 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03d072a8_9433_41a7_89c0_0e7d2acb08e0.slice/crio-5756ecc4190b473d29538804ec3aa3ed61f6e7fba970707dd2c19ac44468f107 WatchSource:0}: Error finding container 5756ecc4190b473d29538804ec3aa3ed61f6e7fba970707dd2c19ac44468f107: Status 404 returned error can't find the container with id 5756ecc4190b473d29538804ec3aa3ed61f6e7fba970707dd2c19ac44468f107 Jul 08 21:34:02.604633 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:02.604598 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f" event={"ID":"03d072a8-9433-41a7-89c0-0e7d2acb08e0","Type":"ContainerStarted","Data":"5756ecc4190b473d29538804ec3aa3ed61f6e7fba970707dd2c19ac44468f107"} Jul 08 21:34:03.014728 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:03.014646 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9gck\" (UniqueName: \"kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck\") pod \"network-check-target-wlk2h\" (UID: \"1dc458d1-00fb-427f-b71b-612cab976ce1\") " pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:34:03.014728 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:03.014703 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs\") pod \"network-metrics-daemon-8bgg4\" (UID: \"57804798-9f60-42ad-82f9-8ff4065af152\") " pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:34:03.015207 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:03.014735 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret\") pod \"global-pull-secret-syncer-jbjl2\" (UID: \"1eac0252-00ff-4edd-a198-158b8a99ba18\") " pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:34:03.017360 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:03.017337 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 21:34:03.018139 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:03.018119 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 21:34:03.018241 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:03.018119 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 21:34:03.027385 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:03.027354 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 21:34:03.027646 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:03.027624 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1eac0252-00ff-4edd-a198-158b8a99ba18-original-pull-secret\") pod \"global-pull-secret-syncer-jbjl2\" (UID: \"1eac0252-00ff-4edd-a198-158b8a99ba18\") " pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:34:03.027789 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:03.027771 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57804798-9f60-42ad-82f9-8ff4065af152-metrics-certs\") pod \"network-metrics-daemon-8bgg4\" (UID: \"57804798-9f60-42ad-82f9-8ff4065af152\") " pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:34:03.037794 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:03.037774 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9gck\" (UniqueName: \"kubernetes.io/projected/1dc458d1-00fb-427f-b71b-612cab976ce1-kube-api-access-j9gck\") pod \"network-check-target-wlk2h\" (UID: \"1dc458d1-00fb-427f-b71b-612cab976ce1\") " pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:34:03.054575 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:03.054549 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:03.118632 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:03.118594 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:03.160596 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:03.160568 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wsbgf\"" Jul 08 21:34:03.168656 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:03.168630 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:34:03.169890 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:03.169872 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jbjl2" Jul 08 21:34:03.180257 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:03.180242 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4v6kt\"" Jul 08 21:34:03.188402 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:03.188383 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bgg4" Jul 08 21:34:03.839025 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:03.838996 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:03.899025 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:03.898995 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:04.209396 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:04.209364 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jbjl2"] Jul 08 21:34:04.213188 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:34:04.213159 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eac0252_00ff_4edd_a198_158b8a99ba18.slice/crio-baca76622b497b66855dc85568819fa6301dbc1e8b1627bd0546b79e756c4761 WatchSource:0}: Error finding container baca76622b497b66855dc85568819fa6301dbc1e8b1627bd0546b79e756c4761: Status 404 returned error can't find the container with id baca76622b497b66855dc85568819fa6301dbc1e8b1627bd0546b79e756c4761 Jul 08 21:34:04.226056 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:04.226017 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8bgg4"] Jul 08 21:34:04.229871 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:34:04.229800 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57804798_9f60_42ad_82f9_8ff4065af152.slice/crio-3439f73f4960968aa05689fe38bf9339e778f702b26a22ebbbc2452f25fa5d83 WatchSource:0}: Error finding container 3439f73f4960968aa05689fe38bf9339e778f702b26a22ebbbc2452f25fa5d83: Status 404 returned error can't find the container with id 3439f73f4960968aa05689fe38bf9339e778f702b26a22ebbbc2452f25fa5d83 Jul 08 21:34:04.394444 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:04.394370 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:04.433213 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:04.433179 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wlk2h"] Jul 08 21:34:04.436041 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:34:04.436012 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dc458d1_00fb_427f_b71b_612cab976ce1.slice/crio-7359b625aa616dd3823689b61e2c43d307b9448a2437d12fe08a72f98375077a WatchSource:0}: Error finding container 7359b625aa616dd3823689b61e2c43d307b9448a2437d12fe08a72f98375077a: Status 404 returned error can't find the container with id 7359b625aa616dd3823689b61e2c43d307b9448a2437d12fe08a72f98375077a Jul 08 21:34:04.613296 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:04.613256 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f" event={"ID":"03d072a8-9433-41a7-89c0-0e7d2acb08e0","Type":"ContainerStarted","Data":"eea52778e89ed56e74408a3d8037397b59620da249e09785dced86dc4d21d6dc"} Jul 08 21:34:04.613451 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:04.613322 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f" event={"ID":"03d072a8-9433-41a7-89c0-0e7d2acb08e0","Type":"ContainerStarted","Data":"fdd0b338840d0d3369c3a7c9a160d47524f0853ee1f914f651e52150f7c6bf7f"} Jul 08 21:34:04.614886 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:04.614844 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8bgg4" event={"ID":"57804798-9f60-42ad-82f9-8ff4065af152","Type":"ContainerStarted","Data":"3439f73f4960968aa05689fe38bf9339e778f702b26a22ebbbc2452f25fa5d83"} Jul 08 21:34:04.617255 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:04.617190 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wlk2h" event={"ID":"1dc458d1-00fb-427f-b71b-612cab976ce1","Type":"ContainerStarted","Data":"7359b625aa616dd3823689b61e2c43d307b9448a2437d12fe08a72f98375077a"} Jul 08 21:34:04.621152 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:04.619606 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dpk92" event={"ID":"c4075449-512f-49ac-9cdf-65c97d14f19a","Type":"ContainerStarted","Data":"114b3ae08dd9313c9e010a45c4d34bdcba2c85c3f2fc95485079b606219a7175"} Jul 08 21:34:04.622834 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:04.622780 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jbjl2" event={"ID":"1eac0252-00ff-4edd-a198-158b8a99ba18","Type":"ContainerStarted","Data":"baca76622b497b66855dc85568819fa6301dbc1e8b1627bd0546b79e756c4761"} Jul 08 21:34:04.631509 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:04.631451 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-dkc9f" podStartSLOduration=2.352730337 podStartE2EDuration="4.631433446s" totalImagesPullingTime="2.278703109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:34:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:04.630014569 +0000 UTC m=+66.927899505" watchObservedRunningTime="2026-07-08 21:34:04.631433446 +0000 UTC m=+66.929318396" Jul 08 21:34:04.649653 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:04.649575 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dpk92" podStartSLOduration=1.790320812 podStartE2EDuration="10.649561983s" totalImagesPullingTime="8.859241171s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:33:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:04.64948134 +0000 UTC m=+66.947366275" watchObservedRunningTime="2026-07-08 21:34:04.649561983 +0000 UTC m=+66.947446919" Jul 08 21:34:05.628560 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:05.628422 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8bgg4" event={"ID":"57804798-9f60-42ad-82f9-8ff4065af152","Type":"ContainerStarted","Data":"8302c5324e07f06fd645179f345c330bac48aa62564d37d7c8d627e982680e6d"} Jul 08 21:34:05.628560 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:05.628487 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8bgg4" event={"ID":"57804798-9f60-42ad-82f9-8ff4065af152","Type":"ContainerStarted","Data":"d785281187d0d126d0231d720d4e6b7430316b1fd6202b2a23f880b4c4cc81dc"} Jul 08 21:34:05.646717 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:05.646670 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8bgg4" podStartSLOduration=66.777578175 podStartE2EDuration="1m7.646655508s" totalImagesPullingTime="869.077333ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:05.645382457 +0000 UTC m=+67.943267393" watchObservedRunningTime="2026-07-08 21:34:05.646655508 +0000 UTC m=+67.944540443" Jul 08 21:34:05.746584 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:05.746552 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:05.851595 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:05.851526 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:05.951086 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:05.951017 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:06.076744 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.076644 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:06.101991 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.101961 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln"] Jul 08 21:34:06.123901 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.123866 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln"] Jul 08 21:34:06.124050 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.123918 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xbxnq"] Jul 08 21:34:06.124109 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.124053 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln" Jul 08 21:34:06.128594 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.128400 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 08 21:34:06.128594 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.128471 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-vlchn\"" Jul 08 21:34:06.130463 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.130443 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 08 21:34:06.140539 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.140028 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.143345 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.143327 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 21:34:06.143505 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.143332 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 21:34:06.144013 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.143860 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 21:34:06.144324 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.144307 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6w4g6\"" Jul 08 21:34:06.180246 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.180213 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:06.241248 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.241161 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-node-exporter-tls\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.241395 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.241245 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-sys\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.241395 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.241287 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fvhc\" (UniqueName: \"kubernetes.io/projected/5ab77262-7491-43ce-bd89-9bdc245161b7-kube-api-access-9fvhc\") pod \"openshift-state-metrics-54cf96dfcf-hjnln\" (UID: \"5ab77262-7491-43ce-bd89-9bdc245161b7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln" Jul 08 21:34:06.241395 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.241305 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbdnf\" (UniqueName: \"kubernetes.io/projected/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-kube-api-access-xbdnf\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.241395 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.241325 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ab77262-7491-43ce-bd89-9bdc245161b7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-hjnln\" (UID: \"5ab77262-7491-43ce-bd89-9bdc245161b7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln" Jul 08 21:34:06.241395 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.241361 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-metrics-client-ca\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.241658 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.241400 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-root\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.241658 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.241423 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.241658 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.241446 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ab77262-7491-43ce-bd89-9bdc245161b7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-hjnln\" (UID: \"5ab77262-7491-43ce-bd89-9bdc245161b7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln" Jul 08 21:34:06.241658 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.241492 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ab77262-7491-43ce-bd89-9bdc245161b7-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-hjnln\" (UID: \"5ab77262-7491-43ce-bd89-9bdc245161b7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln" Jul 08 21:34:06.241658 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.241507 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-node-exporter-accelerators-collector-config\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.241658 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.241527 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-node-exporter-wtmp\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.241658 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.241546 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-node-exporter-textfile\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.247920 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.247886 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:06.342388 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.342356 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-node-exporter-wtmp\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.342553 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.342402 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-node-exporter-textfile\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.342553 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.342432 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-node-exporter-tls\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.342553 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.342465 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-sys\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.342553 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.342493 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fvhc\" (UniqueName: \"kubernetes.io/projected/5ab77262-7491-43ce-bd89-9bdc245161b7-kube-api-access-9fvhc\") pod \"openshift-state-metrics-54cf96dfcf-hjnln\" (UID: \"5ab77262-7491-43ce-bd89-9bdc245161b7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln" Jul 08 21:34:06.342553 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.342519 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbdnf\" (UniqueName: \"kubernetes.io/projected/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-kube-api-access-xbdnf\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.342553 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.342545 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ab77262-7491-43ce-bd89-9bdc245161b7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-hjnln\" (UID: \"5ab77262-7491-43ce-bd89-9bdc245161b7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln" Jul 08 21:34:06.342881 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.342571 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-metrics-client-ca\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.342881 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.342609 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-root\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.342881 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.342643 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.342881 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.342676 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ab77262-7491-43ce-bd89-9bdc245161b7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-hjnln\" (UID: \"5ab77262-7491-43ce-bd89-9bdc245161b7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln" Jul 08 21:34:06.342881 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.342713 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ab77262-7491-43ce-bd89-9bdc245161b7-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-hjnln\" (UID: \"5ab77262-7491-43ce-bd89-9bdc245161b7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln" Jul 08 21:34:06.342881 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.342737 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-node-exporter-accelerators-collector-config\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.343157 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.342960 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-node-exporter-textfile\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.343157 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.342971 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-sys\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.343157 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.342997 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-node-exporter-wtmp\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.343157 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:34:06.343008 2580 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 08 21:34:06.343157 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:34:06.343079 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ab77262-7491-43ce-bd89-9bdc245161b7-openshift-state-metrics-tls podName:5ab77262-7491-43ce-bd89-9bdc245161b7 nodeName:}" failed. No retries permitted until 2026-07-08 21:34:06.843057413 +0000 UTC m=+69.140942346 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5ab77262-7491-43ce-bd89-9bdc245161b7-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-hjnln" (UID: "5ab77262-7491-43ce-bd89-9bdc245161b7") : secret "openshift-state-metrics-tls" not found Jul 08 21:34:06.343416 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.343369 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-node-exporter-accelerators-collector-config\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.343476 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.343441 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-root\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.344514 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.344485 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ab77262-7491-43ce-bd89-9bdc245161b7-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-hjnln\" (UID: \"5ab77262-7491-43ce-bd89-9bdc245161b7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln" Jul 08 21:34:06.344638 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.344576 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-metrics-client-ca\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.346031 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.346009 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.346693 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.346672 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-node-exporter-tls\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.347513 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.347489 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ab77262-7491-43ce-bd89-9bdc245161b7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-hjnln\" (UID: \"5ab77262-7491-43ce-bd89-9bdc245161b7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln" Jul 08 21:34:06.348718 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.348687 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:06.352960 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.352938 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbdnf\" (UniqueName: \"kubernetes.io/projected/595ca2c2-1d86-4fbf-bc62-96fe59b7fa99-kube-api-access-xbdnf\") pod \"node-exporter-xbxnq\" (UID: \"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99\") " pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.353563 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.353517 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fvhc\" (UniqueName: \"kubernetes.io/projected/5ab77262-7491-43ce-bd89-9bdc245161b7-kube-api-access-9fvhc\") pod \"openshift-state-metrics-54cf96dfcf-hjnln\" (UID: \"5ab77262-7491-43ce-bd89-9bdc245161b7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln" Jul 08 21:34:06.448729 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.448686 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:06.450922 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.450899 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xbxnq" Jul 08 21:34:06.493561 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:34:06.493495 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod595ca2c2_1d86_4fbf_bc62_96fe59b7fa99.slice/crio-cd77000462a8b3cf0058e674ad5de59c1d3f28308f9cc7f65fe8237d49c97ef9 WatchSource:0}: Error finding container cd77000462a8b3cf0058e674ad5de59c1d3f28308f9cc7f65fe8237d49c97ef9: Status 404 returned error can't find the container with id cd77000462a8b3cf0058e674ad5de59c1d3f28308f9cc7f65fe8237d49c97ef9 Jul 08 21:34:06.548538 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.548478 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:06.634177 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.634140 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xbxnq" event={"ID":"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99","Type":"ContainerStarted","Data":"cd77000462a8b3cf0058e674ad5de59c1d3f28308f9cc7f65fe8237d49c97ef9"} Jul 08 21:34:06.649247 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.649215 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:06.808983 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.808877 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:06.847961 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.847921 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ab77262-7491-43ce-bd89-9bdc245161b7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-hjnln\" (UID: \"5ab77262-7491-43ce-bd89-9bdc245161b7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln" Jul 08 21:34:06.850691 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:06.850663 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5ab77262-7491-43ce-bd89-9bdc245161b7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-hjnln\" (UID: \"5ab77262-7491-43ce-bd89-9bdc245161b7\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln" Jul 08 21:34:07.035980 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.035943 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln" Jul 08 21:34:07.090076 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.089967 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:07.161266 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.161233 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:07.165341 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.165313 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:34:07.173726 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.173700 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.176094 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.176067 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 21:34:07.176371 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.176352 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 21:34:07.176473 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.176360 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 21:34:07.178533 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.178510 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 21:34:07.178642 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.178569 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 21:34:07.178780 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.178764 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 21:34:07.178861 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.178764 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 21:34:07.178861 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.178769 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 21:34:07.178954 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.178888 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-6q9ds\"" Jul 08 21:34:07.179021 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.178998 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 21:34:07.186693 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.186670 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:34:07.250070 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.250034 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aacc30e8-9af4-43ea-ada9-85fa05d610fc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.250070 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.250070 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aacc30e8-9af4-43ea-ada9-85fa05d610fc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.250292 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.250108 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.250292 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.250197 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.250292 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.250233 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-config-volume\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.250292 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.250271 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aacc30e8-9af4-43ea-ada9-85fa05d610fc-config-out\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.250466 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.250302 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.250466 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.250321 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-web-config\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.250466 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.250341 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.250466 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.250372 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aacc30e8-9af4-43ea-ada9-85fa05d610fc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.250466 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.250391 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.250466 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.250407 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfln8\" (UniqueName: \"kubernetes.io/projected/aacc30e8-9af4-43ea-ada9-85fa05d610fc-kube-api-access-xfln8\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.250724 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.250480 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aacc30e8-9af4-43ea-ada9-85fa05d610fc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.350940 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.350856 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aacc30e8-9af4-43ea-ada9-85fa05d610fc-config-out\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.350940 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.350904 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.350940 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.350935 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-web-config\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.351202 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.350966 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.351202 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.350999 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aacc30e8-9af4-43ea-ada9-85fa05d610fc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.351202 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.351025 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.351202 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.351050 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfln8\" (UniqueName: \"kubernetes.io/projected/aacc30e8-9af4-43ea-ada9-85fa05d610fc-kube-api-access-xfln8\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.351202 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.351094 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aacc30e8-9af4-43ea-ada9-85fa05d610fc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.351202 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.351144 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aacc30e8-9af4-43ea-ada9-85fa05d610fc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.351202 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.351167 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aacc30e8-9af4-43ea-ada9-85fa05d610fc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.351202 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.351198 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.351576 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.351235 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.351576 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.351261 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-config-volume\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.351576 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.351513 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aacc30e8-9af4-43ea-ada9-85fa05d610fc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.352285 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:34:07.351907 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aacc30e8-9af4-43ea-ada9-85fa05d610fc-alertmanager-trusted-ca-bundle podName:aacc30e8-9af4-43ea-ada9-85fa05d610fc nodeName:}" failed. No retries permitted until 2026-07-08 21:34:07.85188095 +0000 UTC m=+70.149765869 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/aacc30e8-9af4-43ea-ada9-85fa05d610fc-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "aacc30e8-9af4-43ea-ada9-85fa05d610fc") : configmap references non-existent config key: ca-bundle.crt Jul 08 21:34:07.353340 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.353309 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aacc30e8-9af4-43ea-ada9-85fa05d610fc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.354501 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.354457 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aacc30e8-9af4-43ea-ada9-85fa05d610fc-config-out\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.354771 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.354750 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.356023 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.355748 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-web-config\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.356023 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.355981 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.356167 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.356034 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.356242 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.356213 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-config-volume\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.356365 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.356341 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.357276 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.357253 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aacc30e8-9af4-43ea-ada9-85fa05d610fc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.357521 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.357498 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aacc30e8-9af4-43ea-ada9-85fa05d610fc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.369092 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.369070 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfln8\" (UniqueName: \"kubernetes.io/projected/aacc30e8-9af4-43ea-ada9-85fa05d610fc-kube-api-access-xfln8\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.383612 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.383587 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:07.855514 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.855473 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aacc30e8-9af4-43ea-ada9-85fa05d610fc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:07.856427 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:07.856385 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aacc30e8-9af4-43ea-ada9-85fa05d610fc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aacc30e8-9af4-43ea-ada9-85fa05d610fc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:08.088986 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:08.088948 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 21:34:08.655618 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:08.655582 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:08.722169 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:08.722098 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:09.043466 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:09.043435 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:09.106927 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:09.106892 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:09.327255 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:09.327182 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln"] Jul 08 21:34:09.330219 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:34:09.330112 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ab77262_7491_43ce_bd89_9bdc245161b7.slice/crio-d7d8a015164c6ed96e912e8239834dacff78c26e8672315e2412d2af3fd64d2e WatchSource:0}: Error finding container d7d8a015164c6ed96e912e8239834dacff78c26e8672315e2412d2af3fd64d2e: Status 404 returned error can't find the container with id d7d8a015164c6ed96e912e8239834dacff78c26e8672315e2412d2af3fd64d2e Jul 08 21:34:09.364923 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:09.362005 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 21:34:09.647288 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:09.647246 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jbjl2" event={"ID":"1eac0252-00ff-4edd-a198-158b8a99ba18","Type":"ContainerStarted","Data":"913e725c89c23b9ce84e7bc1d270dc7f989b987d7891b8760d8121dae9b5dce3"} Jul 08 21:34:09.649282 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:09.649255 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln" event={"ID":"5ab77262-7491-43ce-bd89-9bdc245161b7","Type":"ContainerStarted","Data":"f09b721c0dca6584980fe24034693a985246c9d2e8e621b798f7e2cfd3209a20"} Jul 08 21:34:09.649393 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:09.649314 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln" event={"ID":"5ab77262-7491-43ce-bd89-9bdc245161b7","Type":"ContainerStarted","Data":"d18056104b9b13dbba82d097680c88540049f17e60326dd5dea2b4a71fa7846e"} Jul 08 21:34:09.649393 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:09.649343 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln" event={"ID":"5ab77262-7491-43ce-bd89-9bdc245161b7","Type":"ContainerStarted","Data":"d7d8a015164c6ed96e912e8239834dacff78c26e8672315e2412d2af3fd64d2e"} Jul 08 21:34:09.650607 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:09.650515 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aacc30e8-9af4-43ea-ada9-85fa05d610fc","Type":"ContainerStarted","Data":"fff1124969f10496dac7952a67838ac524030228662cbf58052c30628e9b2698"} Jul 08 21:34:09.652212 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:09.652191 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wlk2h" event={"ID":"1dc458d1-00fb-427f-b71b-612cab976ce1","Type":"ContainerStarted","Data":"7fe56f7077f0c59fb21e4066d0801ed8adf5b10883278c3b7c7e4ce8f6e5c35e"} Jul 08 21:34:09.652343 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:09.652322 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:34:09.662609 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:09.662565 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jbjl2" podStartSLOduration=66.689067001 podStartE2EDuration="1m11.662551898s" totalImagesPullingTime="4.973484897s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:09.661868621 +0000 UTC m=+71.959753553" watchObservedRunningTime="2026-07-08 21:34:09.662551898 +0000 UTC m=+71.960436833" Jul 08 21:34:09.677967 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:09.677916 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wlk2h" podStartSLOduration=66.941436342 podStartE2EDuration="1m11.677899739s" totalImagesPullingTime="4.736463397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:09.676932537 +0000 UTC m=+71.974817473" watchObservedRunningTime="2026-07-08 21:34:09.677899739 +0000 UTC m=+71.975784675" Jul 08 21:34:09.751629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:09.751596 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:10.180487 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.180420 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:10.281166 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.281133 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:10.382468 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.382435 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:10.400149 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.400118 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7dd59dc87b-ng8wx"] Jul 08 21:34:10.403612 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.403592 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.406315 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.406297 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 08 21:34:10.406450 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.406433 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-bxt9p\"" Jul 08 21:34:10.406598 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.406581 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 08 21:34:10.406675 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.406646 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 08 21:34:10.407622 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.407603 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-bh144tftacio7\"" Jul 08 21:34:10.407720 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.407609 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 21:34:10.415300 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.415274 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7dd59dc87b-ng8wx"] Jul 08 21:34:10.451391 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.451316 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:10.477434 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.477400 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d49c0b-7f35-435d-909b-d4509393af21-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.477581 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.477445 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/19d49c0b-7f35-435d-909b-d4509393af21-secret-metrics-server-tls\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.477581 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.477488 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9b4k\" (UniqueName: \"kubernetes.io/projected/19d49c0b-7f35-435d-909b-d4509393af21-kube-api-access-s9b4k\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.477581 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.477524 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19d49c0b-7f35-435d-909b-d4509393af21-client-ca-bundle\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.477716 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.477591 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/19d49c0b-7f35-435d-909b-d4509393af21-metrics-server-audit-profiles\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.477716 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.477613 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/19d49c0b-7f35-435d-909b-d4509393af21-secret-metrics-server-client-certs\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.477716 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.477636 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/19d49c0b-7f35-435d-909b-d4509393af21-audit-log\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.578296 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.578259 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/19d49c0b-7f35-435d-909b-d4509393af21-metrics-server-audit-profiles\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.578296 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.578308 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/19d49c0b-7f35-435d-909b-d4509393af21-secret-metrics-server-client-certs\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.578545 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.578342 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/19d49c0b-7f35-435d-909b-d4509393af21-audit-log\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.578545 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.578370 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d49c0b-7f35-435d-909b-d4509393af21-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.578545 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.578401 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/19d49c0b-7f35-435d-909b-d4509393af21-secret-metrics-server-tls\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.578545 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.578442 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9b4k\" (UniqueName: \"kubernetes.io/projected/19d49c0b-7f35-435d-909b-d4509393af21-kube-api-access-s9b4k\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.578545 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.578476 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19d49c0b-7f35-435d-909b-d4509393af21-client-ca-bundle\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.579206 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.579177 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/19d49c0b-7f35-435d-909b-d4509393af21-audit-log\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.579341 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.579304 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/19d49c0b-7f35-435d-909b-d4509393af21-metrics-server-audit-profiles\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.579790 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.579753 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d49c0b-7f35-435d-909b-d4509393af21-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.581497 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.581465 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/19d49c0b-7f35-435d-909b-d4509393af21-client-ca-bundle\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.581614 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.581535 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/19d49c0b-7f35-435d-909b-d4509393af21-secret-metrics-server-tls\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.582395 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.582376 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/19d49c0b-7f35-435d-909b-d4509393af21-secret-metrics-server-client-certs\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.597731 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.597706 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9b4k\" (UniqueName: \"kubernetes.io/projected/19d49c0b-7f35-435d-909b-d4509393af21-kube-api-access-s9b4k\") pod \"metrics-server-7dd59dc87b-ng8wx\" (UID: \"19d49c0b-7f35-435d-909b-d4509393af21\") " pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.659397 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.659351 2580 generic.go:356] "Generic (PLEG): container finished" podID="595ca2c2-1d86-4fbf-bc62-96fe59b7fa99" containerID="59cd43a4756df1df96a095f64479076e08efc936100e27d7f802af0b0e53ea09" exitCode=0 Jul 08 21:34:10.659540 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.659441 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xbxnq" event={"ID":"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99","Type":"ContainerDied","Data":"59cd43a4756df1df96a095f64479076e08efc936100e27d7f802af0b0e53ea09"} Jul 08 21:34:10.718103 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.718032 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:10.868802 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.868318 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-6mp5c"] Jul 08 21:34:10.870073 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.870039 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:10.871508 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.871489 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6mp5c" Jul 08 21:34:10.873792 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.873775 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 08 21:34:10.873896 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.873805 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-zvr8w\"" Jul 08 21:34:10.883929 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.883904 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-6mp5c"] Jul 08 21:34:10.966153 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.966088 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:10.985433 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.985377 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlwcs\" (UniqueName: \"kubernetes.io/projected/01bee47c-0122-421f-b912-f0d9a8a4efa4-kube-api-access-nlwcs\") pod \"monitoring-plugin-d5dcff7cf-6mp5c\" (UID: \"01bee47c-0122-421f-b912-f0d9a8a4efa4\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6mp5c" Jul 08 21:34:10.985550 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:10.985518 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/01bee47c-0122-421f-b912-f0d9a8a4efa4-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-6mp5c\" (UID: \"01bee47c-0122-421f-b912-f0d9a8a4efa4\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6mp5c" Jul 08 21:34:11.050847 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:11.050799 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7dd59dc87b-ng8wx"] Jul 08 21:34:11.053707 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:11.053678 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:11.073991 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:34:11.073970 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19d49c0b_7f35_435d_909b_d4509393af21.slice/crio-603f6c988499e534fbe7301db24b88a51e2a71bb19cf8e42328d54db94c13fab WatchSource:0}: Error finding container 603f6c988499e534fbe7301db24b88a51e2a71bb19cf8e42328d54db94c13fab: Status 404 returned error can't find the container with id 603f6c988499e534fbe7301db24b88a51e2a71bb19cf8e42328d54db94c13fab Jul 08 21:34:11.085999 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:11.085974 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/01bee47c-0122-421f-b912-f0d9a8a4efa4-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-6mp5c\" (UID: \"01bee47c-0122-421f-b912-f0d9a8a4efa4\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6mp5c" Jul 08 21:34:11.086096 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:11.086038 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlwcs\" (UniqueName: \"kubernetes.io/projected/01bee47c-0122-421f-b912-f0d9a8a4efa4-kube-api-access-nlwcs\") pod \"monitoring-plugin-d5dcff7cf-6mp5c\" (UID: \"01bee47c-0122-421f-b912-f0d9a8a4efa4\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6mp5c" Jul 08 21:34:11.088502 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:11.088482 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/01bee47c-0122-421f-b912-f0d9a8a4efa4-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-6mp5c\" (UID: \"01bee47c-0122-421f-b912-f0d9a8a4efa4\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6mp5c" Jul 08 21:34:11.093385 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:11.093355 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlwcs\" (UniqueName: \"kubernetes.io/projected/01bee47c-0122-421f-b912-f0d9a8a4efa4-kube-api-access-nlwcs\") pod \"monitoring-plugin-d5dcff7cf-6mp5c\" (UID: \"01bee47c-0122-421f-b912-f0d9a8a4efa4\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6mp5c" Jul 08 21:34:11.183002 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:11.182971 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6mp5c" Jul 08 21:34:11.308519 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:11.308483 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-6mp5c"] Jul 08 21:34:11.311341 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:34:11.311313 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01bee47c_0122_421f_b912_f0d9a8a4efa4.slice/crio-e62b1d633888e7d78691ac7f14e86be6e4c8868385e20f913dc6d90e4363fd95 WatchSource:0}: Error finding container e62b1d633888e7d78691ac7f14e86be6e4c8868385e20f913dc6d90e4363fd95: Status 404 returned error can't find the container with id e62b1d633888e7d78691ac7f14e86be6e4c8868385e20f913dc6d90e4363fd95 Jul 08 21:34:11.537861 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:11.537835 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:11.606953 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:11.606928 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cngfd" Jul 08 21:34:11.629026 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:11.628989 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:11.666556 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:11.666014 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xbxnq" event={"ID":"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99","Type":"ContainerStarted","Data":"005717ad63126b35f480aa848642131e2c6040629dbda6c3c11bed34816583a4"} Jul 08 21:34:11.666556 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:11.666074 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xbxnq" event={"ID":"595ca2c2-1d86-4fbf-bc62-96fe59b7fa99","Type":"ContainerStarted","Data":"4f785442d681cca133e6bf3978ec57adaab7b7dd638ba8960225abe71d5bf560"} Jul 08 21:34:11.667706 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:11.667651 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" event={"ID":"19d49c0b-7f35-435d-909b-d4509393af21","Type":"ContainerStarted","Data":"603f6c988499e534fbe7301db24b88a51e2a71bb19cf8e42328d54db94c13fab"} Jul 08 21:34:11.670149 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:11.670123 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln" event={"ID":"5ab77262-7491-43ce-bd89-9bdc245161b7","Type":"ContainerStarted","Data":"3ec7f8ae84fdc399493257acf819d0cec004fb858cead4c83d44274316ef79bb"} Jul 08 21:34:11.672852 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:11.672805 2580 generic.go:356] "Generic (PLEG): container finished" podID="aacc30e8-9af4-43ea-ada9-85fa05d610fc" containerID="f3d99091732a59295ee16bdafa67a6e6b10fd14654ec2c85f9d1bcc8fe46476e" exitCode=0 Jul 08 21:34:11.672963 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:11.672907 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aacc30e8-9af4-43ea-ada9-85fa05d610fc","Type":"ContainerDied","Data":"f3d99091732a59295ee16bdafa67a6e6b10fd14654ec2c85f9d1bcc8fe46476e"} Jul 08 21:34:11.674191 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:11.674167 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6mp5c" event={"ID":"01bee47c-0122-421f-b912-f0d9a8a4efa4","Type":"ContainerStarted","Data":"e62b1d633888e7d78691ac7f14e86be6e4c8868385e20f913dc6d90e4363fd95"} Jul 08 21:34:11.685926 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:11.685884 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xbxnq" podStartSLOduration=2.31467812 podStartE2EDuration="5.685868099s" totalImagesPullingTime="3.299126259s" totalInitContainerRuntime="72.06372ms" isStatefulPod=true podCreationTimestamp="2026-07-08 21:34:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:11.684876906 +0000 UTC m=+73.982761841" watchObservedRunningTime="2026-07-08 21:34:11.685868099 +0000 UTC m=+73.983753036" Jul 08 21:34:11.732356 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:11.732297 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-hjnln" podStartSLOduration=4.302547863 podStartE2EDuration="5.732281237s" totalImagesPullingTime="1.429733374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:34:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:11.730523482 +0000 UTC m=+74.028408418" watchObservedRunningTime="2026-07-08 21:34:11.732281237 +0000 UTC m=+74.030166171" Jul 08 21:34:12.354121 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.351922 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 21:34:12.356300 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.356276 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.361552 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.360285 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 21:34:12.362172 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.362146 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:12.363690 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.362895 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 21:34:12.363690 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.362904 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 21:34:12.363690 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.362924 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 21:34:12.363690 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.362987 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 21:34:12.364071 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.364030 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 21:34:12.364156 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.364136 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6g6qz\"" Jul 08 21:34:12.364281 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.364249 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 21:34:12.364281 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.364274 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-a8eg0blajd2op\"" Jul 08 21:34:12.364534 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.364281 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 21:34:12.364534 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.364431 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 21:34:12.364534 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.364457 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 21:34:12.369265 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.369233 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 21:34:12.374473 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.374438 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 21:34:12.374473 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.374440 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 21:34:12.374745 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.374728 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 21:34:12.387183 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.387160 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 21:34:12.498785 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.498745 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-config\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.498966 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.498798 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.498966 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.498860 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.498966 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.498889 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.498966 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.498916 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.498966 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.498956 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9c3397fa-fea0-4750-907c-aee6f78072a5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.499246 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.499054 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-web-config\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.499246 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.499099 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.499246 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.499131 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.499246 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.499174 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.499246 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.499199 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.499246 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.499225 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.499494 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.499249 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.499494 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.499283 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.499494 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.499309 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9c3397fa-fea0-4750-907c-aee6f78072a5-config-out\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.499494 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.499344 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljxrh\" (UniqueName: \"kubernetes.io/projected/9c3397fa-fea0-4750-907c-aee6f78072a5-kube-api-access-ljxrh\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.499494 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.499377 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.499494 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.499400 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.499494 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.499432 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9c3397fa-fea0-4750-907c-aee6f78072a5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.499494 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.499456 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.600147 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.600116 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9c3397fa-fea0-4750-907c-aee6f78072a5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.600324 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.600162 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-web-config\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.600324 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.600192 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.600324 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.600214 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.600324 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.600242 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.600324 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.600262 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.600601 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.600575 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9c3397fa-fea0-4750-907c-aee6f78072a5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.601315 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.600785 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.601315 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.600847 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.601315 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.600882 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.601315 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.600908 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9c3397fa-fea0-4750-907c-aee6f78072a5-config-out\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.601315 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.600945 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljxrh\" (UniqueName: \"kubernetes.io/projected/9c3397fa-fea0-4750-907c-aee6f78072a5-kube-api-access-ljxrh\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.601315 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.600973 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.601315 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.601000 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.601315 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.601313 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.601800 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.601584 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.601800 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.601636 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9c3397fa-fea0-4750-907c-aee6f78072a5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.601800 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.601671 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.601800 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.601701 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-config\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.601800 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.601730 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.601800 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.601769 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.602142 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.601800 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.602142 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.601858 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.603114 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.602539 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.605187 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.604386 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9c3397fa-fea0-4750-907c-aee6f78072a5-config-out\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.605187 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.604749 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.605187 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.604778 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-web-config\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.605187 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.605171 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.606343 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.606280 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-config\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.606841 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.606791 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.610761 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.610736 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.612462 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.612437 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c3397fa-fea0-4750-907c-aee6f78072a5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.612989 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.612946 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.612989 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.612975 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.613403 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.613360 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.613513 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.613477 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.613786 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.613723 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.615640 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.614120 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9c3397fa-fea0-4750-907c-aee6f78072a5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.615640 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.614354 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9c3397fa-fea0-4750-907c-aee6f78072a5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.616774 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.616699 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljxrh\" (UniqueName: \"kubernetes.io/projected/9c3397fa-fea0-4750-907c-aee6f78072a5-kube-api-access-ljxrh\") pod \"prometheus-k8s-0\" (UID: \"9c3397fa-fea0-4750-907c-aee6f78072a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:12.673696 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:12.673666 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:13.336573 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:13.336539 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 21:34:13.369784 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:13.369753 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:13.424387 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:13.424277 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:13.595210 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:34:13.595176 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c3397fa_fea0_4750_907c_aee6f78072a5.slice/crio-f902972e1fec86b1f8f0332a726074aa82ccc0aa821f43aa2e362078921507f4 WatchSource:0}: Error finding container f902972e1fec86b1f8f0332a726074aa82ccc0aa821f43aa2e362078921507f4: Status 404 returned error can't find the container with id f902972e1fec86b1f8f0332a726074aa82ccc0aa821f43aa2e362078921507f4 Jul 08 21:34:13.685186 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:13.685120 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6mp5c" event={"ID":"01bee47c-0122-421f-b912-f0d9a8a4efa4","Type":"ContainerStarted","Data":"b06ed37373996de97b8d1666b334c4554205c8902d482c5b6654e4fda2fd7f43"} Jul 08 21:34:13.685401 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:13.685380 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6mp5c" Jul 08 21:34:13.687384 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:13.687343 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" event={"ID":"19d49c0b-7f35-435d-909b-d4509393af21","Type":"ContainerStarted","Data":"f44b2e941bc77fe90855fc9e08333018e09d1d5aa8108a404f97e06e3a066621"} Jul 08 21:34:13.690570 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:13.690519 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c3397fa-fea0-4750-907c-aee6f78072a5","Type":"ContainerStarted","Data":"f902972e1fec86b1f8f0332a726074aa82ccc0aa821f43aa2e362078921507f4"} Jul 08 21:34:13.690830 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:13.690795 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6mp5c" Jul 08 21:34:13.702338 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:13.702287 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-6mp5c" podStartSLOduration=1.859218443 podStartE2EDuration="3.702267042s" totalImagesPullingTime="1.843048599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:34:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:13.700840489 +0000 UTC m=+75.998725419" watchObservedRunningTime="2026-07-08 21:34:13.702267042 +0000 UTC m=+76.000151979" Jul 08 21:34:13.746347 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:13.746292 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" podStartSLOduration=1.6637547750000001 podStartE2EDuration="3.746272156s" totalImagesPullingTime="2.082517381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:34:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:13.745570615 +0000 UTC m=+76.043455552" watchObservedRunningTime="2026-07-08 21:34:13.746272156 +0000 UTC m=+76.044157092" Jul 08 21:34:14.698384 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:14.698351 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aacc30e8-9af4-43ea-ada9-85fa05d610fc","Type":"ContainerStarted","Data":"b7c88f87c22bd679685569e17db451364147c4991bc98ec1ea8e480e08f84166"} Jul 08 21:34:14.698763 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:14.698423 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aacc30e8-9af4-43ea-ada9-85fa05d610fc","Type":"ContainerStarted","Data":"ee540439bdf756f5c13d5751d551944c408da22407f9e0e79feb7b194a375cb2"} Jul 08 21:34:14.698763 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:14.698451 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aacc30e8-9af4-43ea-ada9-85fa05d610fc","Type":"ContainerStarted","Data":"475b918032eb46b007217b63ca7063e3945b3df3ee406f6a357ab21245490d06"} Jul 08 21:34:14.698763 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:14.698474 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aacc30e8-9af4-43ea-ada9-85fa05d610fc","Type":"ContainerStarted","Data":"2df92bd667915ff121b286485011dd2050d103dbc83716de30a9163a966d96f5"} Jul 08 21:34:14.698763 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:14.698506 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aacc30e8-9af4-43ea-ada9-85fa05d610fc","Type":"ContainerStarted","Data":"e3eca1d862542b5b69525e17899bc419909eecdff916aea41b9bcd6cc451ca03"} Jul 08 21:34:14.700020 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:14.699996 2580 generic.go:356] "Generic (PLEG): container finished" podID="9c3397fa-fea0-4750-907c-aee6f78072a5" containerID="32dfbe7880580eb19fde7f1b2e4d2b393adb98c7714e211cd381d4208f3f0bc4" exitCode=0 Jul 08 21:34:14.700127 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:14.700080 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c3397fa-fea0-4750-907c-aee6f78072a5","Type":"ContainerDied","Data":"32dfbe7880580eb19fde7f1b2e4d2b393adb98c7714e211cd381d4208f3f0bc4"} Jul 08 21:34:15.213916 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:15.213878 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:15.284384 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:15.284348 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:15.708151 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:15.708102 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aacc30e8-9af4-43ea-ada9-85fa05d610fc","Type":"ContainerStarted","Data":"3c1ea7f92c5f81eb2ead64905fbc4ffd28dc75d26118f52ffb68a0d7755a8810"} Jul 08 21:34:15.739700 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:15.739647 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.445186139 podStartE2EDuration="8.73963123s" totalImagesPullingTime="4.27309781s" totalInitContainerRuntime="21.347281ms" isStatefulPod=false podCreationTimestamp="2026-07-08 21:34:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:15.738539326 +0000 UTC m=+78.036424261" watchObservedRunningTime="2026-07-08 21:34:15.73963123 +0000 UTC m=+78.037516165" Jul 08 21:34:15.926160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:15.926091 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:15.992982 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:15.992947 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:16.099779 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:16.099746 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:16.198900 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:16.198830 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:18.724462 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:18.724431 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c3397fa-fea0-4750-907c-aee6f78072a5","Type":"ContainerStarted","Data":"afae71bf31b033aa494bdf04bd141005ddab7438ff26a53912696ebf354808fb"} Jul 08 21:34:18.724904 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:18.724494 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c3397fa-fea0-4750-907c-aee6f78072a5","Type":"ContainerStarted","Data":"74093161fc10883f6cd7ad1213d6bd9b6e53f9644ce270f516ff8bfa7ea442d2"} Jul 08 21:34:19.217166 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:19.215540 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:19.326787 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:19.326746 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:20.736705 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:20.736675 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c3397fa-fea0-4750-907c-aee6f78072a5","Type":"ContainerStarted","Data":"875ed6491fad1487f86e3d8533dde45481d66152c827b6a260f3cacb78732df2"} Jul 08 21:34:20.737102 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:20.736736 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c3397fa-fea0-4750-907c-aee6f78072a5","Type":"ContainerStarted","Data":"77c646878b60bffa7bbbcd70d3ee48f9944927e48608ab9c56bfb2aec3cc73f8"} Jul 08 21:34:20.737102 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:20.736771 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c3397fa-fea0-4750-907c-aee6f78072a5","Type":"ContainerStarted","Data":"99b9a08d6b4edd5c227915811b51b4aa7e0c06854a9d0bf6f31e1895fcb6b021"} Jul 08 21:34:20.737102 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:20.736798 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9c3397fa-fea0-4750-907c-aee6f78072a5","Type":"ContainerStarted","Data":"622257ad502845d7bfeb3fcaa1b629a6918ccc6ead90de1ed63d35f7a4e27e22"} Jul 08 21:34:20.765396 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:20.765353 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.538782585 podStartE2EDuration="8.7653314s" totalImagesPullingTime="5.192176604s" totalInitContainerRuntime="34.372211ms" isStatefulPod=false podCreationTimestamp="2026-07-08 21:34:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:34:20.76308291 +0000 UTC m=+83.060967859" watchObservedRunningTime="2026-07-08 21:34:20.7653314 +0000 UTC m=+83.063216350" Jul 08 21:34:22.078872 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:22.078832 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:22.192902 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:22.192866 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:22.280976 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:22.280944 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:22.442765 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:22.442675 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:22.544179 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:22.544141 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:22.674100 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:22.674068 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:34:23.726221 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:23.726185 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:23.784629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:23.784596 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:28.252544 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:28.252511 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:28.363024 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:28.362994 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:30.718892 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:30.718856 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:30.719256 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:30.718924 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:37.409640 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:37.409601 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:37.465908 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:37.465874 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:40.662203 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:40.662169 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wlk2h" Jul 08 21:34:45.415602 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:45.415559 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:50.724159 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:50.724127 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:50.728069 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:50.728040 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7dd59dc87b-ng8wx" Jul 08 21:34:53.666901 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:53.666868 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:34:53.728074 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:34:53.728038 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:12.674351 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:35:12.674311 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:35:12.694380 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:35:12.694353 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:35:12.944327 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:35:12.944261 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 21:35:19.263301 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:35:19.263264 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:25.343122 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:35:25.343080 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:25.456645 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:35:25.456611 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:36.031456 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:35:36.031405 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:40.570398 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:35:40.570365 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:40.675871 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:35:40.675833 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:41.282163 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:35:41.282124 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:41.383211 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:35:41.383182 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:41.875276 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:35:41.875247 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:41.934096 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:35:41.934063 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:52.388871 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:35:52.388770 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:52.489428 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:35:52.489395 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:52.545694 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:35:52.545672 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:35:59.885075 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:35:59.885041 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:03.338337 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:36:03.338303 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:03.388839 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:36:03.388789 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:08.657950 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:36:08.657912 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:08.758399 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:36:08.758368 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:08.858822 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:36:08.858792 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:16.191250 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:36:16.190751 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:29.892767 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:36:29.892723 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:29.992978 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:36:29.992945 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:56.537435 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:36:56.537403 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:36:57.360054 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:36:57.360003 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:00.163872 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:00.163835 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:00.264931 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:00.264899 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:01.732742 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:01.732711 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:01.812139 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:01.812099 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:01.930470 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:01.930436 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:02.044996 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:02.044965 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:04.100876 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:04.100847 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:04.509054 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:04.508980 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:06.951225 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:06.951190 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:07.008658 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:07.008631 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:09.913182 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:09.909294 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:10.010166 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:10.010131 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:11.311700 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:11.311669 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:12.016359 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:12.016329 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-v4qj5"] Jul 08 21:37:12.019610 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:12.019590 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-v4qj5" Jul 08 21:37:12.022485 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:12.022467 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-g9bjd\"" Jul 08 21:37:12.022936 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:12.022921 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 21:37:12.023536 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:12.023520 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 21:37:12.029989 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:12.029968 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-v4qj5"] Jul 08 21:37:12.163375 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:12.163343 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48ddf8b0-e99c-48ac-9662-1a1ed0ca8dc3-bound-sa-token\") pod \"cert-manager-55b4d954d5-v4qj5\" (UID: \"48ddf8b0-e99c-48ac-9662-1a1ed0ca8dc3\") " pod="cert-manager/cert-manager-55b4d954d5-v4qj5" Jul 08 21:37:12.163508 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:12.163383 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nznlm\" (UniqueName: \"kubernetes.io/projected/48ddf8b0-e99c-48ac-9662-1a1ed0ca8dc3-kube-api-access-nznlm\") pod \"cert-manager-55b4d954d5-v4qj5\" (UID: \"48ddf8b0-e99c-48ac-9662-1a1ed0ca8dc3\") " pod="cert-manager/cert-manager-55b4d954d5-v4qj5" Jul 08 21:37:12.264622 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:12.264591 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48ddf8b0-e99c-48ac-9662-1a1ed0ca8dc3-bound-sa-token\") pod \"cert-manager-55b4d954d5-v4qj5\" (UID: \"48ddf8b0-e99c-48ac-9662-1a1ed0ca8dc3\") " pod="cert-manager/cert-manager-55b4d954d5-v4qj5" Jul 08 21:37:12.264742 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:12.264628 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nznlm\" (UniqueName: \"kubernetes.io/projected/48ddf8b0-e99c-48ac-9662-1a1ed0ca8dc3-kube-api-access-nznlm\") pod \"cert-manager-55b4d954d5-v4qj5\" (UID: \"48ddf8b0-e99c-48ac-9662-1a1ed0ca8dc3\") " pod="cert-manager/cert-manager-55b4d954d5-v4qj5" Jul 08 21:37:12.273940 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:12.273912 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/48ddf8b0-e99c-48ac-9662-1a1ed0ca8dc3-bound-sa-token\") pod \"cert-manager-55b4d954d5-v4qj5\" (UID: \"48ddf8b0-e99c-48ac-9662-1a1ed0ca8dc3\") " pod="cert-manager/cert-manager-55b4d954d5-v4qj5" Jul 08 21:37:12.276094 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:12.276075 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nznlm\" (UniqueName: \"kubernetes.io/projected/48ddf8b0-e99c-48ac-9662-1a1ed0ca8dc3-kube-api-access-nznlm\") pod \"cert-manager-55b4d954d5-v4qj5\" (UID: \"48ddf8b0-e99c-48ac-9662-1a1ed0ca8dc3\") " pod="cert-manager/cert-manager-55b4d954d5-v4qj5" Jul 08 21:37:12.340216 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:12.340189 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-v4qj5" Jul 08 21:37:12.465939 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:12.465907 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-v4qj5"] Jul 08 21:37:12.469134 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:37:12.469099 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48ddf8b0_e99c_48ac_9662_1a1ed0ca8dc3.slice/crio-fc61538cbfcd1a569ad532cc6aaf66d29c22eacc5c5d9523c033ea837f22ef4e WatchSource:0}: Error finding container fc61538cbfcd1a569ad532cc6aaf66d29c22eacc5c5d9523c033ea837f22ef4e: Status 404 returned error can't find the container with id fc61538cbfcd1a569ad532cc6aaf66d29c22eacc5c5d9523c033ea837f22ef4e Jul 08 21:37:13.353090 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:13.353043 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-v4qj5" event={"ID":"48ddf8b0-e99c-48ac-9662-1a1ed0ca8dc3","Type":"ContainerStarted","Data":"fc61538cbfcd1a569ad532cc6aaf66d29c22eacc5c5d9523c033ea837f22ef4e"} Jul 08 21:37:14.434089 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:14.434056 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:14.530016 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:14.529977 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:16.370992 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:16.370954 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-v4qj5" event={"ID":"48ddf8b0-e99c-48ac-9662-1a1ed0ca8dc3","Type":"ContainerStarted","Data":"0563788883e5209e9fa794d6bd01603c001799b7e8ce82a892f78dce74596e80"} Jul 08 21:37:16.387141 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:16.387092 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-v4qj5" podStartSLOduration=2.120446411 podStartE2EDuration="5.387076783s" totalImagesPullingTime="3.266630372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:37:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:37:16.385931091 +0000 UTC m=+258.683816027" watchObservedRunningTime="2026-07-08 21:37:16.387076783 +0000 UTC m=+258.684961721" Jul 08 21:37:21.351048 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:21.350967 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:21.451196 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:21.451163 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:23.347705 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:23.347673 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:35.707251 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.707222 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:35.765471 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.765443 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-857546b9c4-v886x"] Jul 08 21:37:35.769740 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.769721 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-857546b9c4-v886x" Jul 08 21:37:35.776016 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.775617 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 21:37:35.776016 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.775938 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 21:37:35.776993 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.776970 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 21:37:35.776993 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.776985 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bsxnh\"" Jul 08 21:37:35.777099 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.777056 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 21:37:35.777099 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.777070 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 21:37:35.787904 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.787884 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-857546b9c4-v886x"] Jul 08 21:37:35.846865 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.846838 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aaadcb70-0dcf-48a6-aafa-27bec22ba179-cert\") pod \"lws-controller-manager-857546b9c4-v886x\" (UID: \"aaadcb70-0dcf-48a6-aafa-27bec22ba179\") " pod="openshift-lws-operator/lws-controller-manager-857546b9c4-v886x" Jul 08 21:37:35.847015 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.846885 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w8r5\" (UniqueName: \"kubernetes.io/projected/aaadcb70-0dcf-48a6-aafa-27bec22ba179-kube-api-access-9w8r5\") pod \"lws-controller-manager-857546b9c4-v886x\" (UID: \"aaadcb70-0dcf-48a6-aafa-27bec22ba179\") " pod="openshift-lws-operator/lws-controller-manager-857546b9c4-v886x" Jul 08 21:37:35.847015 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.846975 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aaadcb70-0dcf-48a6-aafa-27bec22ba179-metrics-cert\") pod \"lws-controller-manager-857546b9c4-v886x\" (UID: \"aaadcb70-0dcf-48a6-aafa-27bec22ba179\") " pod="openshift-lws-operator/lws-controller-manager-857546b9c4-v886x" Jul 08 21:37:35.847119 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.847042 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/aaadcb70-0dcf-48a6-aafa-27bec22ba179-manager-config\") pod \"lws-controller-manager-857546b9c4-v886x\" (UID: \"aaadcb70-0dcf-48a6-aafa-27bec22ba179\") " pod="openshift-lws-operator/lws-controller-manager-857546b9c4-v886x" Jul 08 21:37:35.850235 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.850213 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:35.947574 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.947539 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aaadcb70-0dcf-48a6-aafa-27bec22ba179-cert\") pod \"lws-controller-manager-857546b9c4-v886x\" (UID: \"aaadcb70-0dcf-48a6-aafa-27bec22ba179\") " pod="openshift-lws-operator/lws-controller-manager-857546b9c4-v886x" Jul 08 21:37:35.947713 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.947597 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w8r5\" (UniqueName: \"kubernetes.io/projected/aaadcb70-0dcf-48a6-aafa-27bec22ba179-kube-api-access-9w8r5\") pod \"lws-controller-manager-857546b9c4-v886x\" (UID: \"aaadcb70-0dcf-48a6-aafa-27bec22ba179\") " pod="openshift-lws-operator/lws-controller-manager-857546b9c4-v886x" Jul 08 21:37:35.947713 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.947637 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aaadcb70-0dcf-48a6-aafa-27bec22ba179-metrics-cert\") pod \"lws-controller-manager-857546b9c4-v886x\" (UID: \"aaadcb70-0dcf-48a6-aafa-27bec22ba179\") " pod="openshift-lws-operator/lws-controller-manager-857546b9c4-v886x" Jul 08 21:37:35.947713 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.947669 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/aaadcb70-0dcf-48a6-aafa-27bec22ba179-manager-config\") pod \"lws-controller-manager-857546b9c4-v886x\" (UID: \"aaadcb70-0dcf-48a6-aafa-27bec22ba179\") " pod="openshift-lws-operator/lws-controller-manager-857546b9c4-v886x" Jul 08 21:37:35.948377 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.948354 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/aaadcb70-0dcf-48a6-aafa-27bec22ba179-manager-config\") pod \"lws-controller-manager-857546b9c4-v886x\" (UID: \"aaadcb70-0dcf-48a6-aafa-27bec22ba179\") " pod="openshift-lws-operator/lws-controller-manager-857546b9c4-v886x" Jul 08 21:37:35.950192 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.950170 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aaadcb70-0dcf-48a6-aafa-27bec22ba179-metrics-cert\") pod \"lws-controller-manager-857546b9c4-v886x\" (UID: \"aaadcb70-0dcf-48a6-aafa-27bec22ba179\") " pod="openshift-lws-operator/lws-controller-manager-857546b9c4-v886x" Jul 08 21:37:35.950288 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.950243 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aaadcb70-0dcf-48a6-aafa-27bec22ba179-cert\") pod \"lws-controller-manager-857546b9c4-v886x\" (UID: \"aaadcb70-0dcf-48a6-aafa-27bec22ba179\") " pod="openshift-lws-operator/lws-controller-manager-857546b9c4-v886x" Jul 08 21:37:35.966786 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:35.966726 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w8r5\" (UniqueName: \"kubernetes.io/projected/aaadcb70-0dcf-48a6-aafa-27bec22ba179-kube-api-access-9w8r5\") pod \"lws-controller-manager-857546b9c4-v886x\" (UID: \"aaadcb70-0dcf-48a6-aafa-27bec22ba179\") " pod="openshift-lws-operator/lws-controller-manager-857546b9c4-v886x" Jul 08 21:37:36.078668 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:36.078643 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-857546b9c4-v886x" Jul 08 21:37:36.213537 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:36.213501 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-857546b9c4-v886x"] Jul 08 21:37:36.223032 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:37:36.223008 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaadcb70_0dcf_48a6_aafa_27bec22ba179.slice/crio-b1fe8d9eeb9c2f13cbcde234a82f0142299b4ca990587394c1d747c14f899aaa WatchSource:0}: Error finding container b1fe8d9eeb9c2f13cbcde234a82f0142299b4ca990587394c1d747c14f899aaa: Status 404 returned error can't find the container with id b1fe8d9eeb9c2f13cbcde234a82f0142299b4ca990587394c1d747c14f899aaa Jul 08 21:37:36.441249 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:36.441218 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-857546b9c4-v886x" event={"ID":"aaadcb70-0dcf-48a6-aafa-27bec22ba179","Type":"ContainerStarted","Data":"b1fe8d9eeb9c2f13cbcde234a82f0142299b4ca990587394c1d747c14f899aaa"} Jul 08 21:37:36.998292 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:36.998253 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:38.033319 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.033291 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c"] Jul 08 21:37:38.039861 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.039830 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" Jul 08 21:37:38.043763 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.042607 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 21:37:38.043763 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.042887 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 21:37:38.043763 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.043589 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 21:37:38.043763 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.043658 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 21:37:38.043763 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.043722 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-trgck\"" Jul 08 21:37:38.053903 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.053862 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c"] Jul 08 21:37:38.067563 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.067498 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78ea6aaf-1175-495d-8d84-21178e0b833c-tmp\") pod \"opendatahub-operator-controller-manager-c97f5c75f-jct7c\" (UID: \"78ea6aaf-1175-495d-8d84-21178e0b833c\") " pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" Jul 08 21:37:38.067689 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.067570 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/78ea6aaf-1175-495d-8d84-21178e0b833c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c97f5c75f-jct7c\" (UID: \"78ea6aaf-1175-495d-8d84-21178e0b833c\") " pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" Jul 08 21:37:38.067689 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.067637 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnjxt\" (UniqueName: \"kubernetes.io/projected/78ea6aaf-1175-495d-8d84-21178e0b833c-kube-api-access-xnjxt\") pod \"opendatahub-operator-controller-manager-c97f5c75f-jct7c\" (UID: \"78ea6aaf-1175-495d-8d84-21178e0b833c\") " pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" Jul 08 21:37:38.067822 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.067735 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/78ea6aaf-1175-495d-8d84-21178e0b833c-webhook-cert\") pod \"opendatahub-operator-controller-manager-c97f5c75f-jct7c\" (UID: \"78ea6aaf-1175-495d-8d84-21178e0b833c\") " pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" Jul 08 21:37:38.067822 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.067766 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/78ea6aaf-1175-495d-8d84-21178e0b833c-manifests\") pod \"opendatahub-operator-controller-manager-c97f5c75f-jct7c\" (UID: \"78ea6aaf-1175-495d-8d84-21178e0b833c\") " pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" Jul 08 21:37:38.169198 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.169164 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/78ea6aaf-1175-495d-8d84-21178e0b833c-webhook-cert\") pod \"opendatahub-operator-controller-manager-c97f5c75f-jct7c\" (UID: \"78ea6aaf-1175-495d-8d84-21178e0b833c\") " pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" Jul 08 21:37:38.169393 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.169221 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/78ea6aaf-1175-495d-8d84-21178e0b833c-manifests\") pod \"opendatahub-operator-controller-manager-c97f5c75f-jct7c\" (UID: \"78ea6aaf-1175-495d-8d84-21178e0b833c\") " pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" Jul 08 21:37:38.169393 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.169285 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78ea6aaf-1175-495d-8d84-21178e0b833c-tmp\") pod \"opendatahub-operator-controller-manager-c97f5c75f-jct7c\" (UID: \"78ea6aaf-1175-495d-8d84-21178e0b833c\") " pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" Jul 08 21:37:38.169393 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.169311 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/78ea6aaf-1175-495d-8d84-21178e0b833c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c97f5c75f-jct7c\" (UID: \"78ea6aaf-1175-495d-8d84-21178e0b833c\") " pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" Jul 08 21:37:38.169393 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.169360 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnjxt\" (UniqueName: \"kubernetes.io/projected/78ea6aaf-1175-495d-8d84-21178e0b833c-kube-api-access-xnjxt\") pod \"opendatahub-operator-controller-manager-c97f5c75f-jct7c\" (UID: \"78ea6aaf-1175-495d-8d84-21178e0b833c\") " pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" Jul 08 21:37:38.169756 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.169718 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/78ea6aaf-1175-495d-8d84-21178e0b833c-manifests\") pod \"opendatahub-operator-controller-manager-c97f5c75f-jct7c\" (UID: \"78ea6aaf-1175-495d-8d84-21178e0b833c\") " pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" Jul 08 21:37:38.172283 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.172247 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78ea6aaf-1175-495d-8d84-21178e0b833c-tmp\") pod \"opendatahub-operator-controller-manager-c97f5c75f-jct7c\" (UID: \"78ea6aaf-1175-495d-8d84-21178e0b833c\") " pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" Jul 08 21:37:38.172412 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.172387 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/78ea6aaf-1175-495d-8d84-21178e0b833c-webhook-cert\") pod \"opendatahub-operator-controller-manager-c97f5c75f-jct7c\" (UID: \"78ea6aaf-1175-495d-8d84-21178e0b833c\") " pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" Jul 08 21:37:38.172519 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.172429 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/78ea6aaf-1175-495d-8d84-21178e0b833c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c97f5c75f-jct7c\" (UID: \"78ea6aaf-1175-495d-8d84-21178e0b833c\") " pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" Jul 08 21:37:38.181673 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.181648 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnjxt\" (UniqueName: \"kubernetes.io/projected/78ea6aaf-1175-495d-8d84-21178e0b833c-kube-api-access-xnjxt\") pod \"opendatahub-operator-controller-manager-c97f5c75f-jct7c\" (UID: \"78ea6aaf-1175-495d-8d84-21178e0b833c\") " pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" Jul 08 21:37:38.355589 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.355510 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" Jul 08 21:37:38.662390 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.662367 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:38.676878 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:38.676854 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c"] Jul 08 21:37:38.679385 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:37:38.679283 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78ea6aaf_1175_495d_8d84_21178e0b833c.slice/crio-07844fe73258ae608b31ed7f74b8d14c857e0f30b799218bea1b971787d9fcfd WatchSource:0}: Error finding container 07844fe73258ae608b31ed7f74b8d14c857e0f30b799218bea1b971787d9fcfd: Status 404 returned error can't find the container with id 07844fe73258ae608b31ed7f74b8d14c857e0f30b799218bea1b971787d9fcfd Jul 08 21:37:39.455010 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:39.454966 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-857546b9c4-v886x" event={"ID":"aaadcb70-0dcf-48a6-aafa-27bec22ba179","Type":"ContainerStarted","Data":"3a689b59649a394c4b98b022ea8586edb338afe59d3a0ba68d8a487c46e7a05d"} Jul 08 21:37:39.455571 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:39.455089 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-857546b9c4-v886x" Jul 08 21:37:39.456605 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:39.456574 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" event={"ID":"78ea6aaf-1175-495d-8d84-21178e0b833c","Type":"ContainerStarted","Data":"07844fe73258ae608b31ed7f74b8d14c857e0f30b799218bea1b971787d9fcfd"} Jul 08 21:37:39.473165 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:39.473120 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-857546b9c4-v886x" podStartSLOduration=2.113565552 podStartE2EDuration="4.473107908s" totalImagesPullingTime="2.359542356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:37:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:37:39.472051926 +0000 UTC m=+281.769936862" watchObservedRunningTime="2026-07-08 21:37:39.473107908 +0000 UTC m=+281.770992842" Jul 08 21:37:41.465083 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:41.465047 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" event={"ID":"78ea6aaf-1175-495d-8d84-21178e0b833c","Type":"ContainerStarted","Data":"06f03bc1674a169e95a0cf899da57e95dee423376366f4cf28b3041697572a3b"} Jul 08 21:37:43.473318 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:43.473283 2580 generic.go:356] "Generic (PLEG): container finished" podID="78ea6aaf-1175-495d-8d84-21178e0b833c" containerID="06f03bc1674a169e95a0cf899da57e95dee423376366f4cf28b3041697572a3b" exitCode=0 Jul 08 21:37:43.473716 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:43.473329 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" event={"ID":"78ea6aaf-1175-495d-8d84-21178e0b833c","Type":"ContainerDied","Data":"06f03bc1674a169e95a0cf899da57e95dee423376366f4cf28b3041697572a3b"} Jul 08 21:37:44.478803 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:44.478767 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" event={"ID":"78ea6aaf-1175-495d-8d84-21178e0b833c","Type":"ContainerStarted","Data":"0fbe5612fcbb2ba02025677526bd78d6ba96a3cdd6fbc33137c0d43517855516"} Jul 08 21:37:44.479151 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:44.479032 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" Jul 08 21:37:44.503865 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:44.503791 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" podStartSLOduration=2.386655561 podStartE2EDuration="6.503772354s" totalImagesPullingTime="3.029759165s" totalInitContainerRuntime="1.087357628s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:37:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:37:44.502111406 +0000 UTC m=+286.799996341" watchObservedRunningTime="2026-07-08 21:37:44.503772354 +0000 UTC m=+286.801657289" Jul 08 21:37:49.995560 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:49.995523 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:50.211672 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:50.211637 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:50.462686 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:50.462653 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-857546b9c4-v886x" Jul 08 21:37:55.484952 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:55.484920 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-c97f5c75f-jct7c" Jul 08 21:37:56.978740 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:56.978710 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:57.149515 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.149481 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:57.617961 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.617930 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:57.684226 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.684194 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-589c8d496f-grdph"] Jul 08 21:37:57.687766 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.687745 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-589c8d496f-grdph" Jul 08 21:37:57.690088 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.690062 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 21:37:57.690204 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.690106 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 08 21:37:57.691077 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.691054 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 21:37:57.691180 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.691123 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xcd4c\"" Jul 08 21:37:57.691180 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.691156 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 21:37:57.696176 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.696154 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-589c8d496f-grdph"] Jul 08 21:37:57.816974 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.816946 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:57.840012 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.839982 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e2dd961-2fb7-4da7-b915-bef8ff7825f6-tmp\") pod \"kube-auth-proxy-589c8d496f-grdph\" (UID: \"8e2dd961-2fb7-4da7-b915-bef8ff7825f6\") " pod="openshift-ingress/kube-auth-proxy-589c8d496f-grdph" Jul 08 21:37:57.840125 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.840035 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e2dd961-2fb7-4da7-b915-bef8ff7825f6-tls-certs\") pod \"kube-auth-proxy-589c8d496f-grdph\" (UID: \"8e2dd961-2fb7-4da7-b915-bef8ff7825f6\") " pod="openshift-ingress/kube-auth-proxy-589c8d496f-grdph" Jul 08 21:37:57.840164 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.840123 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62bv2\" (UniqueName: \"kubernetes.io/projected/8e2dd961-2fb7-4da7-b915-bef8ff7825f6-kube-api-access-62bv2\") pod \"kube-auth-proxy-589c8d496f-grdph\" (UID: \"8e2dd961-2fb7-4da7-b915-bef8ff7825f6\") " pod="openshift-ingress/kube-auth-proxy-589c8d496f-grdph" Jul 08 21:37:57.941160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.941084 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e2dd961-2fb7-4da7-b915-bef8ff7825f6-tls-certs\") pod \"kube-auth-proxy-589c8d496f-grdph\" (UID: \"8e2dd961-2fb7-4da7-b915-bef8ff7825f6\") " pod="openshift-ingress/kube-auth-proxy-589c8d496f-grdph" Jul 08 21:37:57.941282 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.941161 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62bv2\" (UniqueName: \"kubernetes.io/projected/8e2dd961-2fb7-4da7-b915-bef8ff7825f6-kube-api-access-62bv2\") pod \"kube-auth-proxy-589c8d496f-grdph\" (UID: \"8e2dd961-2fb7-4da7-b915-bef8ff7825f6\") " pod="openshift-ingress/kube-auth-proxy-589c8d496f-grdph" Jul 08 21:37:57.941282 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.941193 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e2dd961-2fb7-4da7-b915-bef8ff7825f6-tmp\") pod \"kube-auth-proxy-589c8d496f-grdph\" (UID: \"8e2dd961-2fb7-4da7-b915-bef8ff7825f6\") " pod="openshift-ingress/kube-auth-proxy-589c8d496f-grdph" Jul 08 21:37:57.943289 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.943262 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e2dd961-2fb7-4da7-b915-bef8ff7825f6-tmp\") pod \"kube-auth-proxy-589c8d496f-grdph\" (UID: \"8e2dd961-2fb7-4da7-b915-bef8ff7825f6\") " pod="openshift-ingress/kube-auth-proxy-589c8d496f-grdph" Jul 08 21:37:57.943631 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.943607 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e2dd961-2fb7-4da7-b915-bef8ff7825f6-tls-certs\") pod \"kube-auth-proxy-589c8d496f-grdph\" (UID: \"8e2dd961-2fb7-4da7-b915-bef8ff7825f6\") " pod="openshift-ingress/kube-auth-proxy-589c8d496f-grdph" Jul 08 21:37:57.952378 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.952352 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62bv2\" (UniqueName: \"kubernetes.io/projected/8e2dd961-2fb7-4da7-b915-bef8ff7825f6-kube-api-access-62bv2\") pod \"kube-auth-proxy-589c8d496f-grdph\" (UID: \"8e2dd961-2fb7-4da7-b915-bef8ff7825f6\") " pod="openshift-ingress/kube-auth-proxy-589c8d496f-grdph" Jul 08 21:37:57.979533 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.979496 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:57.998618 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:57.998595 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-589c8d496f-grdph" Jul 08 21:37:58.074158 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:58.074129 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:58.137107 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:58.137077 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-589c8d496f-grdph"] Jul 08 21:37:58.139730 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:37:58.139702 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e2dd961_2fb7_4da7_b915_bef8ff7825f6.slice/crio-a08d2c400d843e9ce4eae168754c212bc5f4a027563586aaa57be26cc293265d WatchSource:0}: Error finding container a08d2c400d843e9ce4eae168754c212bc5f4a027563586aaa57be26cc293265d: Status 404 returned error can't find the container with id a08d2c400d843e9ce4eae168754c212bc5f4a027563586aaa57be26cc293265d Jul 08 21:37:58.144498 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:58.144480 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 21:37:58.171858 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:58.171830 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:58.533103 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:58.533066 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-589c8d496f-grdph" event={"ID":"8e2dd961-2fb7-4da7-b915-bef8ff7825f6","Type":"ContainerStarted","Data":"a08d2c400d843e9ce4eae168754c212bc5f4a027563586aaa57be26cc293265d"} Jul 08 21:37:58.742284 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:58.742244 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:58.939706 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:58.939619 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:59.024337 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:59.024296 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:37:59.941719 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:37:59.941682 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:00.775244 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:00.775224 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 21:38:01.546483 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:01.546447 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-589c8d496f-grdph" event={"ID":"8e2dd961-2fb7-4da7-b915-bef8ff7825f6","Type":"ContainerStarted","Data":"fa1eba758f2f7f0c2fff6f90f8bcb4b85d9b7dacc6db87533ed945f2c321b133"} Jul 08 21:38:01.562629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:01.562584 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-589c8d496f-grdph" podStartSLOduration=1.931485608 podStartE2EDuration="4.562569556s" totalImagesPullingTime="2.631083948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:38:01.561098139 +0000 UTC m=+303.858983087" watchObservedRunningTime="2026-07-08 21:38:01.562569556 +0000 UTC m=+303.860454492" Jul 08 21:38:04.699070 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:04.699032 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:04.817040 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:04.817010 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:05.007117 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:05.007048 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:05.132863 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:05.132832 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:05.259272 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:05.259200 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:07.878119 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:07.878080 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:07.955737 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:07.955707 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2kc5f"] Jul 08 21:38:07.962317 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:07.962298 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2kc5f" Jul 08 21:38:07.965896 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:07.965878 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-qgwc9\"" Jul 08 21:38:07.965995 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:07.965893 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 08 21:38:07.971361 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:07.971336 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2kc5f"] Jul 08 21:38:07.985729 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:07.985699 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:08.127806 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:08.127773 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/57934621-1051-4944-9faf-d715c06d948a-cert\") pod \"kserve-controller-manager-d6f4b65b8-2kc5f\" (UID: \"57934621-1051-4944-9faf-d715c06d948a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2kc5f" Jul 08 21:38:08.127985 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:08.127893 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qvzm\" (UniqueName: \"kubernetes.io/projected/57934621-1051-4944-9faf-d715c06d948a-kube-api-access-9qvzm\") pod \"kserve-controller-manager-d6f4b65b8-2kc5f\" (UID: \"57934621-1051-4944-9faf-d715c06d948a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2kc5f" Jul 08 21:38:08.134508 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:08.134442 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:08.228791 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:08.228755 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qvzm\" (UniqueName: \"kubernetes.io/projected/57934621-1051-4944-9faf-d715c06d948a-kube-api-access-9qvzm\") pod \"kserve-controller-manager-d6f4b65b8-2kc5f\" (UID: \"57934621-1051-4944-9faf-d715c06d948a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2kc5f" Jul 08 21:38:08.228949 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:08.228856 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/57934621-1051-4944-9faf-d715c06d948a-cert\") pod \"kserve-controller-manager-d6f4b65b8-2kc5f\" (UID: \"57934621-1051-4944-9faf-d715c06d948a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2kc5f" Jul 08 21:38:08.231357 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:08.231336 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/57934621-1051-4944-9faf-d715c06d948a-cert\") pod \"kserve-controller-manager-d6f4b65b8-2kc5f\" (UID: \"57934621-1051-4944-9faf-d715c06d948a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2kc5f" Jul 08 21:38:08.246483 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:08.246454 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qvzm\" (UniqueName: \"kubernetes.io/projected/57934621-1051-4944-9faf-d715c06d948a-kube-api-access-9qvzm\") pod \"kserve-controller-manager-d6f4b65b8-2kc5f\" (UID: \"57934621-1051-4944-9faf-d715c06d948a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2kc5f" Jul 08 21:38:08.272926 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:08.272900 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2kc5f" Jul 08 21:38:08.398178 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:08.398096 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:08.424055 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:08.424026 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2kc5f"] Jul 08 21:38:08.426632 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:38:08.426609 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57934621_1051_4944_9faf_d715c06d948a.slice/crio-e05c8811bd2e616ee5433a206780f6000dce904c77bf85e283345115ffe3ca7c WatchSource:0}: Error finding container e05c8811bd2e616ee5433a206780f6000dce904c77bf85e283345115ffe3ca7c: Status 404 returned error can't find the container with id e05c8811bd2e616ee5433a206780f6000dce904c77bf85e283345115ffe3ca7c Jul 08 21:38:08.427988 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:08.427973 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 21:38:08.574657 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:08.574623 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2kc5f" event={"ID":"57934621-1051-4944-9faf-d715c06d948a","Type":"ContainerStarted","Data":"e05c8811bd2e616ee5433a206780f6000dce904c77bf85e283345115ffe3ca7c"} Jul 08 21:38:09.187728 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:09.187692 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:09.311268 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:09.311216 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:09.439522 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:09.439441 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:11.595782 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:11.595746 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2kc5f" event={"ID":"57934621-1051-4944-9faf-d715c06d948a","Type":"ContainerStarted","Data":"cced9da6981105d646378e66c53c826df8e36f46b18536ce80cef8ce170a0104"} Jul 08 21:38:11.596163 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:11.595895 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2kc5f" Jul 08 21:38:11.615235 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:11.615186 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2kc5f" podStartSLOduration=2.058839772 podStartE2EDuration="4.615171948s" totalImagesPullingTime="2.556332176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:38:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:38:11.614038374 +0000 UTC m=+313.911923308" watchObservedRunningTime="2026-07-08 21:38:11.615171948 +0000 UTC m=+313.913056920" Jul 08 21:38:42.608311 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:42.608277 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2kc5f" Jul 08 21:38:58.597272 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:58.597176 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:58.695711 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:58.695658 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:58.793421 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:58.793388 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:59.521320 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:59.521290 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:38:59.718891 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:38:59.718845 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:28.884055 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:28.884024 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:28.985328 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:28.985296 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:30.833357 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:30.833302 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:30.892216 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:30.892187 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:33.516722 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:33.516688 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:36.713242 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:36.713205 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:41.559771 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:41.559735 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:41.785915 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:41.785882 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:42.111526 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:42.111478 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:43.574992 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:43.574957 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:45.961377 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:45.961344 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:46.201100 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:46.201067 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:46.366721 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:46.366690 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:48.131992 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:48.131956 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptbrj"] Jul 08 21:39:48.135680 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:48.135655 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptbrj" Jul 08 21:39:48.138281 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:48.138255 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-ct4fx\"" Jul 08 21:39:48.139291 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:48.139273 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 21:39:48.139393 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:48.139359 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 21:39:48.139458 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:48.139373 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 08 21:39:48.150876 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:48.150855 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptbrj"] Jul 08 21:39:48.185147 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:48.185101 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:48.268159 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:48.268129 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r5f6\" (UniqueName: \"kubernetes.io/projected/a4e7dc17-0bbe-45ef-8ee9-c4f47f3bcd01-kube-api-access-5r5f6\") pod \"dns-operator-controller-manager-864dd78c4c-ptbrj\" (UID: \"a4e7dc17-0bbe-45ef-8ee9-c4f47f3bcd01\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptbrj" Jul 08 21:39:48.368882 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:48.368853 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r5f6\" (UniqueName: \"kubernetes.io/projected/a4e7dc17-0bbe-45ef-8ee9-c4f47f3bcd01-kube-api-access-5r5f6\") pod \"dns-operator-controller-manager-864dd78c4c-ptbrj\" (UID: \"a4e7dc17-0bbe-45ef-8ee9-c4f47f3bcd01\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptbrj" Jul 08 21:39:48.377388 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:48.377359 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r5f6\" (UniqueName: \"kubernetes.io/projected/a4e7dc17-0bbe-45ef-8ee9-c4f47f3bcd01-kube-api-access-5r5f6\") pod \"dns-operator-controller-manager-864dd78c4c-ptbrj\" (UID: \"a4e7dc17-0bbe-45ef-8ee9-c4f47f3bcd01\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptbrj" Jul 08 21:39:48.445147 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:48.445052 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptbrj" Jul 08 21:39:48.574594 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:48.574564 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptbrj"] Jul 08 21:39:48.577270 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:39:48.577240 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4e7dc17_0bbe_45ef_8ee9_c4f47f3bcd01.slice/crio-6f0f4cfd963e55a3a0218b46baebf9d681ca749fe0cb995ed0fe811ba39b9fa0 WatchSource:0}: Error finding container 6f0f4cfd963e55a3a0218b46baebf9d681ca749fe0cb995ed0fe811ba39b9fa0: Status 404 returned error can't find the container with id 6f0f4cfd963e55a3a0218b46baebf9d681ca749fe0cb995ed0fe811ba39b9fa0 Jul 08 21:39:49.013191 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:49.013154 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptbrj" event={"ID":"a4e7dc17-0bbe-45ef-8ee9-c4f47f3bcd01","Type":"ContainerStarted","Data":"6f0f4cfd963e55a3a0218b46baebf9d681ca749fe0cb995ed0fe811ba39b9fa0"} Jul 08 21:39:49.916520 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:49.916483 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:50.019315 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:50.019282 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:51.023468 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:51.023432 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptbrj" event={"ID":"a4e7dc17-0bbe-45ef-8ee9-c4f47f3bcd01","Type":"ContainerStarted","Data":"69a8e2d19bbae55794b7136fa209bb1973e7b5434cb5a461335f659d4019e9dc"} Jul 08 21:39:51.023907 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:51.023533 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptbrj" Jul 08 21:39:51.040986 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:51.040935 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptbrj" podStartSLOduration=0.769432467 podStartE2EDuration="3.040920542s" totalImagesPullingTime="2.271488075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:39:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:39:51.038707051 +0000 UTC m=+413.336591987" watchObservedRunningTime="2026-07-08 21:39:51.040920542 +0000 UTC m=+413.338805478" Jul 08 21:39:51.707474 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:51.707443 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-sct7s"] Jul 08 21:39:51.710632 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:51.710618 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sct7s" Jul 08 21:39:51.715091 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:51.715066 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 08 21:39:51.715207 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:51.715069 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 08 21:39:51.715207 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:51.715177 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gm7tq\"" Jul 08 21:39:51.720899 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:51.720880 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-sct7s"] Jul 08 21:39:51.803367 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:51.803335 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/df2fd060-9a73-4605-a4a4-f2fa62d61be6-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-sct7s\" (UID: \"df2fd060-9a73-4605-a4a4-f2fa62d61be6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sct7s" Jul 08 21:39:51.803530 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:51.803384 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4g25\" (UniqueName: \"kubernetes.io/projected/df2fd060-9a73-4605-a4a4-f2fa62d61be6-kube-api-access-p4g25\") pod \"kuadrant-console-plugin-66b5774985-sct7s\" (UID: \"df2fd060-9a73-4605-a4a4-f2fa62d61be6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sct7s" Jul 08 21:39:51.803530 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:51.803478 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/df2fd060-9a73-4605-a4a4-f2fa62d61be6-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-sct7s\" (UID: \"df2fd060-9a73-4605-a4a4-f2fa62d61be6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sct7s" Jul 08 21:39:51.854101 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:51.854045 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:51.904697 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:51.904667 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/df2fd060-9a73-4605-a4a4-f2fa62d61be6-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-sct7s\" (UID: \"df2fd060-9a73-4605-a4a4-f2fa62d61be6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sct7s" Jul 08 21:39:51.904863 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:51.904706 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4g25\" (UniqueName: \"kubernetes.io/projected/df2fd060-9a73-4605-a4a4-f2fa62d61be6-kube-api-access-p4g25\") pod \"kuadrant-console-plugin-66b5774985-sct7s\" (UID: \"df2fd060-9a73-4605-a4a4-f2fa62d61be6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sct7s" Jul 08 21:39:51.904863 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:51.904847 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/df2fd060-9a73-4605-a4a4-f2fa62d61be6-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-sct7s\" (UID: \"df2fd060-9a73-4605-a4a4-f2fa62d61be6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sct7s" Jul 08 21:39:51.904996 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:39:51.904850 2580 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 08 21:39:51.904996 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:39:51.904978 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df2fd060-9a73-4605-a4a4-f2fa62d61be6-plugin-serving-cert podName:df2fd060-9a73-4605-a4a4-f2fa62d61be6 nodeName:}" failed. No retries permitted until 2026-07-08 21:39:52.404959682 +0000 UTC m=+414.702844596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/df2fd060-9a73-4605-a4a4-f2fa62d61be6-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-sct7s" (UID: "df2fd060-9a73-4605-a4a4-f2fa62d61be6") : secret "plugin-serving-cert" not found Jul 08 21:39:51.905436 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:51.905410 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/df2fd060-9a73-4605-a4a4-f2fa62d61be6-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-sct7s\" (UID: \"df2fd060-9a73-4605-a4a4-f2fa62d61be6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sct7s" Jul 08 21:39:51.929550 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:51.929525 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4g25\" (UniqueName: \"kubernetes.io/projected/df2fd060-9a73-4605-a4a4-f2fa62d61be6-kube-api-access-p4g25\") pod \"kuadrant-console-plugin-66b5774985-sct7s\" (UID: \"df2fd060-9a73-4605-a4a4-f2fa62d61be6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sct7s" Jul 08 21:39:52.086091 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:52.086055 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:52.410722 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:52.410632 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/df2fd060-9a73-4605-a4a4-f2fa62d61be6-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-sct7s\" (UID: \"df2fd060-9a73-4605-a4a4-f2fa62d61be6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sct7s" Jul 08 21:39:52.413222 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:52.413200 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/df2fd060-9a73-4605-a4a4-f2fa62d61be6-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-sct7s\" (UID: \"df2fd060-9a73-4605-a4a4-f2fa62d61be6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sct7s" Jul 08 21:39:52.619899 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:52.619862 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sct7s" Jul 08 21:39:52.754195 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:52.754172 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-sct7s"] Jul 08 21:39:52.756007 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:39:52.755979 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf2fd060_9a73_4605_a4a4_f2fa62d61be6.slice/crio-789c70475d4676273f37c5f34ca691f4964e0cafb40801f78dbc3b19e3a6a99f WatchSource:0}: Error finding container 789c70475d4676273f37c5f34ca691f4964e0cafb40801f78dbc3b19e3a6a99f: Status 404 returned error can't find the container with id 789c70475d4676273f37c5f34ca691f4964e0cafb40801f78dbc3b19e3a6a99f Jul 08 21:39:53.033121 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:53.033090 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sct7s" event={"ID":"df2fd060-9a73-4605-a4a4-f2fa62d61be6","Type":"ContainerStarted","Data":"789c70475d4676273f37c5f34ca691f4964e0cafb40801f78dbc3b19e3a6a99f"} Jul 08 21:39:53.329017 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:53.328941 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:39:53.429106 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:39:53.429076 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:01.890100 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:01.890069 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:02.029936 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:02.029908 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ptbrj" Jul 08 21:40:02.063970 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:02.063921 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:03.613143 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:03.613102 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h"] Jul 08 21:40:03.657657 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:03.657624 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h"] Jul 08 21:40:03.657849 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:03.657723 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h" Jul 08 21:40:03.662553 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:03.662530 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-r8m52\"" Jul 08 21:40:03.714092 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:03.714047 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ljj5\" (UniqueName: \"kubernetes.io/projected/dc58366d-8699-4d53-bf18-dd95f0714264-kube-api-access-5ljj5\") pod \"kuadrant-operator-controller-manager-86878c769f-cbp2h\" (UID: \"dc58366d-8699-4d53-bf18-dd95f0714264\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h" Jul 08 21:40:03.714244 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:03.714107 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc58366d-8699-4d53-bf18-dd95f0714264-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cbp2h\" (UID: \"dc58366d-8699-4d53-bf18-dd95f0714264\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h" Jul 08 21:40:03.814554 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:03.814514 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ljj5\" (UniqueName: \"kubernetes.io/projected/dc58366d-8699-4d53-bf18-dd95f0714264-kube-api-access-5ljj5\") pod \"kuadrant-operator-controller-manager-86878c769f-cbp2h\" (UID: \"dc58366d-8699-4d53-bf18-dd95f0714264\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h" Jul 08 21:40:03.814736 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:03.814580 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc58366d-8699-4d53-bf18-dd95f0714264-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cbp2h\" (UID: \"dc58366d-8699-4d53-bf18-dd95f0714264\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h" Jul 08 21:40:03.815033 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:03.815009 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc58366d-8699-4d53-bf18-dd95f0714264-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cbp2h\" (UID: \"dc58366d-8699-4d53-bf18-dd95f0714264\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h" Jul 08 21:40:03.825717 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:03.825689 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ljj5\" (UniqueName: \"kubernetes.io/projected/dc58366d-8699-4d53-bf18-dd95f0714264-kube-api-access-5ljj5\") pod \"kuadrant-operator-controller-manager-86878c769f-cbp2h\" (UID: \"dc58366d-8699-4d53-bf18-dd95f0714264\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h" Jul 08 21:40:03.970714 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:03.970623 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h" Jul 08 21:40:04.275707 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:04.275675 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:04.379441 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:04.379407 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:08.609313 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:08.609278 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:08.751022 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:08.750956 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:11.350975 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:11.350892 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:11.453109 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:11.453073 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:14.474028 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:14.473999 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h"] Jul 08 21:40:14.475402 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:40:14.475381 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc58366d_8699_4d53_bf18_dd95f0714264.slice/crio-8a10bfa433f17bba8ae116632ec38f3e8edc3efcbb8311e6398af8294291cc29 WatchSource:0}: Error finding container 8a10bfa433f17bba8ae116632ec38f3e8edc3efcbb8311e6398af8294291cc29: Status 404 returned error can't find the container with id 8a10bfa433f17bba8ae116632ec38f3e8edc3efcbb8311e6398af8294291cc29 Jul 08 21:40:15.141325 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:15.141290 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sct7s" event={"ID":"df2fd060-9a73-4605-a4a4-f2fa62d61be6","Type":"ContainerStarted","Data":"b96bb35b35f59132809316adda0479b7fc2cee8de29a1b9584e7bb7dd80f3080"} Jul 08 21:40:15.142629 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:15.142602 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h" event={"ID":"dc58366d-8699-4d53-bf18-dd95f0714264","Type":"ContainerStarted","Data":"8a10bfa433f17bba8ae116632ec38f3e8edc3efcbb8311e6398af8294291cc29"} Jul 08 21:40:15.177749 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:15.177692 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sct7s" podStartSLOduration=2.51747707 podStartE2EDuration="24.17767135s" totalImagesPullingTime="21.66019428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:39:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:40:15.177004893 +0000 UTC m=+437.474889830" watchObservedRunningTime="2026-07-08 21:40:15.17767135 +0000 UTC m=+437.475556286" Jul 08 21:40:19.987198 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:19.987090 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:20.087259 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:20.087223 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:21.174072 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:21.174035 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h" event={"ID":"dc58366d-8699-4d53-bf18-dd95f0714264","Type":"ContainerStarted","Data":"3c33ed40e84eeb408a461d941135a280a2997f7a152c6b5d8a2bb892b5c1f9dc"} Jul 08 21:40:21.174455 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:21.174166 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h" Jul 08 21:40:21.218067 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:21.218019 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h" podStartSLOduration=12.095281866 podStartE2EDuration="18.218003407s" totalImagesPullingTime="6.122721541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:40:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:40:21.215505663 +0000 UTC m=+443.513390597" watchObservedRunningTime="2026-07-08 21:40:21.218003407 +0000 UTC m=+443.515888341" Jul 08 21:40:22.437741 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:22.437703 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:22.497469 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:22.497419 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:24.470391 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:24.470361 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:24.531160 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:24.531131 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:32.180957 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:32.180927 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h" Jul 08 21:40:36.285334 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:36.285293 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:36.337448 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:36.337416 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:45.572538 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:45.572497 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:45.628336 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:45.628303 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:48.766334 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:48.766288 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:49.206009 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:49.205974 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:54.727092 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:54.727056 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:54.856777 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:54.856745 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:54.974741 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:54.974709 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:55.712331 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:55.712301 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:40:55.813038 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:40:55.812994 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:14.333141 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:41:14.333079 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:14.432570 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:41:14.432532 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:14.837799 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:41:14.837763 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:27.571638 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:41:27.571602 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:27.695312 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:41:27.695254 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:33.630300 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:41:33.630270 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:33.727227 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:41:33.727190 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:33.781482 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:41:33.781454 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:33.940281 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:41:33.940202 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:37.021791 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:41:37.021754 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:41:37.123872 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:41:37.123834 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:07.532949 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:07.532920 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:08.130315 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:08.130286 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:08.637868 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:08.637827 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:08.743955 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:08.743915 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:19.587786 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:19.587749 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:19.689148 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:19.689109 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:21.729682 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:21.729652 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:21.832006 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:21.831971 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:28.744368 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:28.744334 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:37.157741 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:37.157710 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:37.255705 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:37.255671 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:40.283258 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.283228 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx"] Jul 08 21:42:40.286200 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.286183 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:40.290531 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.290507 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 21:42:40.290531 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.290523 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 21:42:40.291864 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.291843 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mwth9\"" Jul 08 21:42:40.291947 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.291843 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 08 21:42:40.298232 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.298210 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx"] Jul 08 21:42:40.347267 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.347241 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3cf3621-a61b-4181-a7b4-cc7480788086-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jqbzx\" (UID: \"d3cf3621-a61b-4181-a7b4-cc7480788086\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:40.347445 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.347307 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3cf3621-a61b-4181-a7b4-cc7480788086-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jqbzx\" (UID: \"d3cf3621-a61b-4181-a7b4-cc7480788086\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:40.347445 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.347331 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3cf3621-a61b-4181-a7b4-cc7480788086-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jqbzx\" (UID: \"d3cf3621-a61b-4181-a7b4-cc7480788086\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:40.347842 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.347786 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mszc\" (UniqueName: \"kubernetes.io/projected/d3cf3621-a61b-4181-a7b4-cc7480788086-kube-api-access-4mszc\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jqbzx\" (UID: \"d3cf3621-a61b-4181-a7b4-cc7480788086\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:40.347955 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.347867 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3cf3621-a61b-4181-a7b4-cc7480788086-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jqbzx\" (UID: \"d3cf3621-a61b-4181-a7b4-cc7480788086\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:40.347955 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.347885 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3cf3621-a61b-4181-a7b4-cc7480788086-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jqbzx\" (UID: \"d3cf3621-a61b-4181-a7b4-cc7480788086\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:40.389528 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.389504 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:42:40.448392 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.448359 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mszc\" (UniqueName: \"kubernetes.io/projected/d3cf3621-a61b-4181-a7b4-cc7480788086-kube-api-access-4mszc\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jqbzx\" (UID: \"d3cf3621-a61b-4181-a7b4-cc7480788086\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:40.448560 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.448396 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3cf3621-a61b-4181-a7b4-cc7480788086-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jqbzx\" (UID: \"d3cf3621-a61b-4181-a7b4-cc7480788086\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:40.448560 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.448413 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3cf3621-a61b-4181-a7b4-cc7480788086-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jqbzx\" (UID: \"d3cf3621-a61b-4181-a7b4-cc7480788086\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:40.448560 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.448448 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3cf3621-a61b-4181-a7b4-cc7480788086-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jqbzx\" (UID: \"d3cf3621-a61b-4181-a7b4-cc7480788086\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:40.448560 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.448483 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3cf3621-a61b-4181-a7b4-cc7480788086-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jqbzx\" (UID: \"d3cf3621-a61b-4181-a7b4-cc7480788086\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:40.448560 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.448503 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3cf3621-a61b-4181-a7b4-cc7480788086-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jqbzx\" (UID: \"d3cf3621-a61b-4181-a7b4-cc7480788086\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:40.448890 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.448868 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3cf3621-a61b-4181-a7b4-cc7480788086-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jqbzx\" (UID: \"d3cf3621-a61b-4181-a7b4-cc7480788086\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:40.448890 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.448881 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3cf3621-a61b-4181-a7b4-cc7480788086-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jqbzx\" (UID: \"d3cf3621-a61b-4181-a7b4-cc7480788086\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:40.448971 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.448923 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3cf3621-a61b-4181-a7b4-cc7480788086-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jqbzx\" (UID: \"d3cf3621-a61b-4181-a7b4-cc7480788086\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:40.450969 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.450950 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3cf3621-a61b-4181-a7b4-cc7480788086-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jqbzx\" (UID: \"d3cf3621-a61b-4181-a7b4-cc7480788086\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:40.451091 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.451076 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3cf3621-a61b-4181-a7b4-cc7480788086-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jqbzx\" (UID: \"d3cf3621-a61b-4181-a7b4-cc7480788086\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:40.456410 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.456387 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mszc\" (UniqueName: \"kubernetes.io/projected/d3cf3621-a61b-4181-a7b4-cc7480788086-kube-api-access-4mszc\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jqbzx\" (UID: \"d3cf3621-a61b-4181-a7b4-cc7480788086\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:40.596352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.596261 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:40.732302 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.732277 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx"] Jul 08 21:42:40.734415 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:42:40.734381 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3cf3621_a61b_4181_a7b4_cc7480788086.slice/crio-e27b12e6ff913001a0167f64df4f920408ada4549685e6e0fab8ed1e1c06bba1 WatchSource:0}: Error finding container e27b12e6ff913001a0167f64df4f920408ada4549685e6e0fab8ed1e1c06bba1: Status 404 returned error can't find the container with id e27b12e6ff913001a0167f64df4f920408ada4549685e6e0fab8ed1e1c06bba1 Jul 08 21:42:40.774187 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:40.774152 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" event={"ID":"d3cf3621-a61b-4181-a7b4-cc7480788086","Type":"ContainerStarted","Data":"e27b12e6ff913001a0167f64df4f920408ada4549685e6e0fab8ed1e1c06bba1"} Jul 08 21:42:42.783323 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:42.783286 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" event={"ID":"d3cf3621-a61b-4181-a7b4-cc7480788086","Type":"ContainerStarted","Data":"caa12a8269337f18eba3494f87afdf291d48e0075ef083fcc6aed56099ade5db"} Jul 08 21:42:42.803141 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:42.803097 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" podStartSLOduration=1.795061207 podStartE2EDuration="2.803081024s" totalImagesPullingTime="1.008019817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:42:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:42:42.801614344 +0000 UTC m=+585.099499280" watchObservedRunningTime="2026-07-08 21:42:42.803081024 +0000 UTC m=+585.100965988" Jul 08 21:42:50.596897 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:50.596865 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:50.597284 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:50.596922 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:50.609355 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:50.609333 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:50.830470 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:50.830444 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jqbzx" Jul 08 21:42:53.686002 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:42:53.685967 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:43:02.680322 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:02.680291 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:43:05.373590 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.373554 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb"] Jul 08 21:43:05.376839 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.376801 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:05.379334 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.379318 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 08 21:43:05.388308 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.388287 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb"] Jul 08 21:43:05.473923 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.473892 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/005898a5-8bda-4179-b470-c8d8ac5a869b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhpzb\" (UID: \"005898a5-8bda-4179-b470-c8d8ac5a869b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:05.473923 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.473922 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/005898a5-8bda-4179-b470-c8d8ac5a869b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhpzb\" (UID: \"005898a5-8bda-4179-b470-c8d8ac5a869b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:05.474114 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.473963 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwlsm\" (UniqueName: \"kubernetes.io/projected/005898a5-8bda-4179-b470-c8d8ac5a869b-kube-api-access-pwlsm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhpzb\" (UID: \"005898a5-8bda-4179-b470-c8d8ac5a869b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:05.474114 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.474024 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/005898a5-8bda-4179-b470-c8d8ac5a869b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhpzb\" (UID: \"005898a5-8bda-4179-b470-c8d8ac5a869b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:05.474114 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.474056 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/005898a5-8bda-4179-b470-c8d8ac5a869b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhpzb\" (UID: \"005898a5-8bda-4179-b470-c8d8ac5a869b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:05.474114 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.474082 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/005898a5-8bda-4179-b470-c8d8ac5a869b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhpzb\" (UID: \"005898a5-8bda-4179-b470-c8d8ac5a869b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:05.574974 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.574945 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/005898a5-8bda-4179-b470-c8d8ac5a869b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhpzb\" (UID: \"005898a5-8bda-4179-b470-c8d8ac5a869b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:05.575134 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.574978 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/005898a5-8bda-4179-b470-c8d8ac5a869b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhpzb\" (UID: \"005898a5-8bda-4179-b470-c8d8ac5a869b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:05.575134 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.575006 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/005898a5-8bda-4179-b470-c8d8ac5a869b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhpzb\" (UID: \"005898a5-8bda-4179-b470-c8d8ac5a869b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:05.575134 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.575022 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/005898a5-8bda-4179-b470-c8d8ac5a869b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhpzb\" (UID: \"005898a5-8bda-4179-b470-c8d8ac5a869b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:05.575134 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.575060 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwlsm\" (UniqueName: \"kubernetes.io/projected/005898a5-8bda-4179-b470-c8d8ac5a869b-kube-api-access-pwlsm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhpzb\" (UID: \"005898a5-8bda-4179-b470-c8d8ac5a869b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:05.575134 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.575101 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/005898a5-8bda-4179-b470-c8d8ac5a869b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhpzb\" (UID: \"005898a5-8bda-4179-b470-c8d8ac5a869b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:05.575392 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.575329 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/005898a5-8bda-4179-b470-c8d8ac5a869b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhpzb\" (UID: \"005898a5-8bda-4179-b470-c8d8ac5a869b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:05.575455 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.575389 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/005898a5-8bda-4179-b470-c8d8ac5a869b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhpzb\" (UID: \"005898a5-8bda-4179-b470-c8d8ac5a869b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:05.575498 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.575470 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/005898a5-8bda-4179-b470-c8d8ac5a869b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhpzb\" (UID: \"005898a5-8bda-4179-b470-c8d8ac5a869b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:05.577441 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.577420 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/005898a5-8bda-4179-b470-c8d8ac5a869b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhpzb\" (UID: \"005898a5-8bda-4179-b470-c8d8ac5a869b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:05.577563 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.577545 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/005898a5-8bda-4179-b470-c8d8ac5a869b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhpzb\" (UID: \"005898a5-8bda-4179-b470-c8d8ac5a869b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:05.579587 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.579567 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:43:05.582615 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.582595 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwlsm\" (UniqueName: \"kubernetes.io/projected/005898a5-8bda-4179-b470-c8d8ac5a869b-kube-api-access-pwlsm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lhpzb\" (UID: \"005898a5-8bda-4179-b470-c8d8ac5a869b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:05.687084 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.687015 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:05.821669 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.821647 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb"] Jul 08 21:43:05.824327 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:43:05.824302 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod005898a5_8bda_4179_b470_c8d8ac5a869b.slice/crio-3e06a1436009290bab65f2af5b35a59596093d298f8c77ef8d01dab185dd1022 WatchSource:0}: Error finding container 3e06a1436009290bab65f2af5b35a59596093d298f8c77ef8d01dab185dd1022: Status 404 returned error can't find the container with id 3e06a1436009290bab65f2af5b35a59596093d298f8c77ef8d01dab185dd1022 Jul 08 21:43:05.890103 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:05.890066 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" event={"ID":"005898a5-8bda-4179-b470-c8d8ac5a869b","Type":"ContainerStarted","Data":"3e06a1436009290bab65f2af5b35a59596093d298f8c77ef8d01dab185dd1022"} Jul 08 21:43:06.896245 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:06.896209 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" event={"ID":"005898a5-8bda-4179-b470-c8d8ac5a869b","Type":"ContainerStarted","Data":"eb9743ea7ac0da2b2cc2888da396163ccacd39c42bfd82bde7989a78e0367bca"} Jul 08 21:43:06.916230 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:06.916176 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" podStartSLOduration=1.742898847 podStartE2EDuration="1.916155607s" totalImagesPullingTime="173.25676ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:43:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:43:06.915032858 +0000 UTC m=+609.212917792" watchObservedRunningTime="2026-07-08 21:43:06.916155607 +0000 UTC m=+609.214040543" Jul 08 21:43:15.687352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:15.687302 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:15.687843 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:15.687399 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:15.699994 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:15.699970 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:15.956873 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:15.956762 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lhpzb" Jul 08 21:43:18.403469 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.403386 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk"] Jul 08 21:43:18.407026 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.407008 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:18.409375 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.409354 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 08 21:43:18.417402 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.417382 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk"] Jul 08 21:43:18.437159 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.437136 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:43:18.483341 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.483314 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx5d5\" (UniqueName: \"kubernetes.io/projected/26f60c46-b8a9-48d5-9691-3c2e78df8767-kube-api-access-rx5d5\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk\" (UID: \"26f60c46-b8a9-48d5-9691-3c2e78df8767\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:18.483460 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.483343 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26f60c46-b8a9-48d5-9691-3c2e78df8767-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk\" (UID: \"26f60c46-b8a9-48d5-9691-3c2e78df8767\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:18.483460 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.483363 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26f60c46-b8a9-48d5-9691-3c2e78df8767-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk\" (UID: \"26f60c46-b8a9-48d5-9691-3c2e78df8767\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:18.483460 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.483420 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26f60c46-b8a9-48d5-9691-3c2e78df8767-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk\" (UID: \"26f60c46-b8a9-48d5-9691-3c2e78df8767\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:18.483623 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.483533 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26f60c46-b8a9-48d5-9691-3c2e78df8767-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk\" (UID: \"26f60c46-b8a9-48d5-9691-3c2e78df8767\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:18.483623 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.483560 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26f60c46-b8a9-48d5-9691-3c2e78df8767-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk\" (UID: \"26f60c46-b8a9-48d5-9691-3c2e78df8767\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:18.584614 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.584579 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26f60c46-b8a9-48d5-9691-3c2e78df8767-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk\" (UID: \"26f60c46-b8a9-48d5-9691-3c2e78df8767\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:18.584614 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.584620 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26f60c46-b8a9-48d5-9691-3c2e78df8767-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk\" (UID: \"26f60c46-b8a9-48d5-9691-3c2e78df8767\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:18.584858 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.584668 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26f60c46-b8a9-48d5-9691-3c2e78df8767-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk\" (UID: \"26f60c46-b8a9-48d5-9691-3c2e78df8767\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:18.584858 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.584684 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26f60c46-b8a9-48d5-9691-3c2e78df8767-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk\" (UID: \"26f60c46-b8a9-48d5-9691-3c2e78df8767\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:18.584858 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.584722 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx5d5\" (UniqueName: \"kubernetes.io/projected/26f60c46-b8a9-48d5-9691-3c2e78df8767-kube-api-access-rx5d5\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk\" (UID: \"26f60c46-b8a9-48d5-9691-3c2e78df8767\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:18.584858 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.584747 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26f60c46-b8a9-48d5-9691-3c2e78df8767-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk\" (UID: \"26f60c46-b8a9-48d5-9691-3c2e78df8767\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:18.585065 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.585044 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26f60c46-b8a9-48d5-9691-3c2e78df8767-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk\" (UID: \"26f60c46-b8a9-48d5-9691-3c2e78df8767\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:18.585130 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.585104 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26f60c46-b8a9-48d5-9691-3c2e78df8767-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk\" (UID: \"26f60c46-b8a9-48d5-9691-3c2e78df8767\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:18.585185 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.585166 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26f60c46-b8a9-48d5-9691-3c2e78df8767-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk\" (UID: \"26f60c46-b8a9-48d5-9691-3c2e78df8767\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:18.587215 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.587187 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26f60c46-b8a9-48d5-9691-3c2e78df8767-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk\" (UID: \"26f60c46-b8a9-48d5-9691-3c2e78df8767\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:18.587359 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.587342 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26f60c46-b8a9-48d5-9691-3c2e78df8767-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk\" (UID: \"26f60c46-b8a9-48d5-9691-3c2e78df8767\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:18.592279 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.592257 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx5d5\" (UniqueName: \"kubernetes.io/projected/26f60c46-b8a9-48d5-9691-3c2e78df8767-kube-api-access-rx5d5\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk\" (UID: \"26f60c46-b8a9-48d5-9691-3c2e78df8767\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:18.716071 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.716005 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:18.862236 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.862201 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk"] Jul 08 21:43:18.866315 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:43:18.866277 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26f60c46_b8a9_48d5_9691_3c2e78df8767.slice/crio-edebf4b85e245add3689a7f67e5db838d7279d765e5e8dbb86cf5020b37c86ac WatchSource:0}: Error finding container edebf4b85e245add3689a7f67e5db838d7279d765e5e8dbb86cf5020b37c86ac: Status 404 returned error can't find the container with id edebf4b85e245add3689a7f67e5db838d7279d765e5e8dbb86cf5020b37c86ac Jul 08 21:43:18.867964 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.867946 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 21:43:18.960094 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:18.960060 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" event={"ID":"26f60c46-b8a9-48d5-9691-3c2e78df8767","Type":"ContainerStarted","Data":"edebf4b85e245add3689a7f67e5db838d7279d765e5e8dbb86cf5020b37c86ac"} Jul 08 21:43:19.966455 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:19.966418 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" event={"ID":"26f60c46-b8a9-48d5-9691-3c2e78df8767","Type":"ContainerStarted","Data":"9e3dadf52d707315e6cb9d4337f258667aea632cdeef95ae2f73e8aa850b39de"} Jul 08 21:43:28.716417 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:28.716380 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:28.716793 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:28.716433 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:28.728528 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:28.728504 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:28.746507 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:28.746462 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" podStartSLOduration=10.580701547 podStartE2EDuration="10.746446552s" totalImagesPullingTime="165.745005ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:43:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:43:19.988740703 +0000 UTC m=+622.286625641" watchObservedRunningTime="2026-07-08 21:43:28.746446552 +0000 UTC m=+631.044331490" Jul 08 21:43:29.016189 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:29.016107 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk" Jul 08 21:43:39.459352 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:43:39.459321 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:44:06.358415 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:44:06.358384 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:44:16.850793 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:44:16.850752 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:44:16.954495 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:44:16.954447 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:44:18.136768 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:44:18.136734 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:44:18.235983 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:44:18.235952 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:44:31.583108 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:44:31.583075 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:45:11.385127 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:45:11.385084 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:45:28.538922 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:45:28.538889 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:45:30.272717 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:45:30.272662 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:46:11.026198 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:46:11.026167 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:46:11.127023 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:46:11.126988 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:46:11.375626 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:46:11.375537 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:46:17.181314 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:46:17.181279 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:46:17.257038 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:46:17.257004 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:46:17.838319 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:46:17.838284 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:46:17.939417 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:46:17.939385 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:46:23.524617 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:46:23.524535 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:46:23.620830 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:46:23.620782 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:46:34.566592 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:46:34.566561 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:46:34.665567 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:46:34.665531 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:48:48.239898 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:48:48.239855 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:48:49.708871 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:48:49.708833 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:50:13.492901 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:50:13.492864 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h"] Jul 08 21:50:13.493451 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:50:13.493069 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h" podUID="dc58366d-8699-4d53-bf18-dd95f0714264" containerName="manager" containerID="cri-o://3c33ed40e84eeb408a461d941135a280a2997f7a152c6b5d8a2bb892b5c1f9dc" gracePeriod=10 Jul 08 21:50:16.441953 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:50:16.441925 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h" Jul 08 21:50:16.596395 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:50:16.596367 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ljj5\" (UniqueName: \"kubernetes.io/projected/dc58366d-8699-4d53-bf18-dd95f0714264-kube-api-access-5ljj5\") pod \"dc58366d-8699-4d53-bf18-dd95f0714264\" (UID: \"dc58366d-8699-4d53-bf18-dd95f0714264\") " Jul 08 21:50:16.596549 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:50:16.596420 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc58366d-8699-4d53-bf18-dd95f0714264-extensions-socket-volume\") pod \"dc58366d-8699-4d53-bf18-dd95f0714264\" (UID: \"dc58366d-8699-4d53-bf18-dd95f0714264\") " Jul 08 21:50:16.596801 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:50:16.596777 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc58366d-8699-4d53-bf18-dd95f0714264-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dc58366d-8699-4d53-bf18-dd95f0714264" (UID: "dc58366d-8699-4d53-bf18-dd95f0714264"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 21:50:16.598588 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:50:16.598569 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc58366d-8699-4d53-bf18-dd95f0714264-kube-api-access-5ljj5" (OuterVolumeSpecName: "kube-api-access-5ljj5") pod "dc58366d-8699-4d53-bf18-dd95f0714264" (UID: "dc58366d-8699-4d53-bf18-dd95f0714264"). InnerVolumeSpecName "kube-api-access-5ljj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 21:50:16.697174 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:50:16.697137 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ljj5\" (UniqueName: \"kubernetes.io/projected/dc58366d-8699-4d53-bf18-dd95f0714264-kube-api-access-5ljj5\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 21:50:16.697174 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:50:16.697173 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc58366d-8699-4d53-bf18-dd95f0714264-extensions-socket-volume\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 21:50:16.815769 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:50:16.815734 2580 generic.go:356] "Generic (PLEG): container finished" podID="dc58366d-8699-4d53-bf18-dd95f0714264" containerID="3c33ed40e84eeb408a461d941135a280a2997f7a152c6b5d8a2bb892b5c1f9dc" exitCode=0 Jul 08 21:50:16.815927 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:50:16.815782 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h" Jul 08 21:50:16.815927 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:50:16.815798 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h" event={"ID":"dc58366d-8699-4d53-bf18-dd95f0714264","Type":"ContainerDied","Data":"3c33ed40e84eeb408a461d941135a280a2997f7a152c6b5d8a2bb892b5c1f9dc"} Jul 08 21:50:16.815927 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:50:16.815861 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h" event={"ID":"dc58366d-8699-4d53-bf18-dd95f0714264","Type":"ContainerDied","Data":"8a10bfa433f17bba8ae116632ec38f3e8edc3efcbb8311e6398af8294291cc29"} Jul 08 21:50:16.815927 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:50:16.815895 2580 scope.go:117] "RemoveContainer" containerID="3c33ed40e84eeb408a461d941135a280a2997f7a152c6b5d8a2bb892b5c1f9dc" Jul 08 21:50:16.825215 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:50:16.825199 2580 scope.go:117] "RemoveContainer" containerID="3c33ed40e84eeb408a461d941135a280a2997f7a152c6b5d8a2bb892b5c1f9dc" Jul 08 21:50:16.825464 ip-10-0-139-229 kubenswrapper[2580]: E0708 21:50:16.825445 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c33ed40e84eeb408a461d941135a280a2997f7a152c6b5d8a2bb892b5c1f9dc\": container with ID starting with 3c33ed40e84eeb408a461d941135a280a2997f7a152c6b5d8a2bb892b5c1f9dc not found: ID does not exist" containerID="3c33ed40e84eeb408a461d941135a280a2997f7a152c6b5d8a2bb892b5c1f9dc" Jul 08 21:50:16.825526 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:50:16.825473 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c33ed40e84eeb408a461d941135a280a2997f7a152c6b5d8a2bb892b5c1f9dc"} err="failed to get container status \"3c33ed40e84eeb408a461d941135a280a2997f7a152c6b5d8a2bb892b5c1f9dc\": rpc error: code = NotFound desc = could not find container \"3c33ed40e84eeb408a461d941135a280a2997f7a152c6b5d8a2bb892b5c1f9dc\": container with ID starting with 3c33ed40e84eeb408a461d941135a280a2997f7a152c6b5d8a2bb892b5c1f9dc not found: ID does not exist" Jul 08 21:50:16.837177 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:50:16.837153 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h"] Jul 08 21:50:16.842156 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:50:16.842138 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cbp2h"] Jul 08 21:50:18.242446 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:50:18.242411 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc58366d-8699-4d53-bf18-dd95f0714264" path="/var/lib/kubelet/pods/dc58366d-8699-4d53-bf18-dd95f0714264/volumes" Jul 08 21:51:14.695012 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:51:14.694966 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jvvrx"] Jul 08 21:51:14.695414 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:51:14.695305 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc58366d-8699-4d53-bf18-dd95f0714264" containerName="manager" Jul 08 21:51:14.695414 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:51:14.695316 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc58366d-8699-4d53-bf18-dd95f0714264" containerName="manager" Jul 08 21:51:14.695414 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:51:14.695369 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc58366d-8699-4d53-bf18-dd95f0714264" containerName="manager" Jul 08 21:51:14.699595 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:51:14.699580 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jvvrx" Jul 08 21:51:14.702157 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:51:14.702137 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-r8m52\"" Jul 08 21:51:14.709206 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:51:14.709179 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jvvrx"] Jul 08 21:51:14.792288 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:51:14.792260 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tllqw\" (UniqueName: \"kubernetes.io/projected/2dbb2389-5909-4750-99bc-8dfdc353d20e-kube-api-access-tllqw\") pod \"kuadrant-operator-controller-manager-86878c769f-jvvrx\" (UID: \"2dbb2389-5909-4750-99bc-8dfdc353d20e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jvvrx" Jul 08 21:51:14.792449 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:51:14.792318 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2dbb2389-5909-4750-99bc-8dfdc353d20e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jvvrx\" (UID: \"2dbb2389-5909-4750-99bc-8dfdc353d20e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jvvrx" Jul 08 21:51:14.892766 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:51:14.892737 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2dbb2389-5909-4750-99bc-8dfdc353d20e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jvvrx\" (UID: \"2dbb2389-5909-4750-99bc-8dfdc353d20e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jvvrx" Jul 08 21:51:14.892948 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:51:14.892833 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tllqw\" (UniqueName: \"kubernetes.io/projected/2dbb2389-5909-4750-99bc-8dfdc353d20e-kube-api-access-tllqw\") pod \"kuadrant-operator-controller-manager-86878c769f-jvvrx\" (UID: \"2dbb2389-5909-4750-99bc-8dfdc353d20e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jvvrx" Jul 08 21:51:14.893153 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:51:14.893133 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2dbb2389-5909-4750-99bc-8dfdc353d20e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jvvrx\" (UID: \"2dbb2389-5909-4750-99bc-8dfdc353d20e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jvvrx" Jul 08 21:51:14.901391 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:51:14.901364 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tllqw\" (UniqueName: \"kubernetes.io/projected/2dbb2389-5909-4750-99bc-8dfdc353d20e-kube-api-access-tllqw\") pod \"kuadrant-operator-controller-manager-86878c769f-jvvrx\" (UID: \"2dbb2389-5909-4750-99bc-8dfdc353d20e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jvvrx" Jul 08 21:51:15.010699 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:51:15.010609 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jvvrx" Jul 08 21:51:15.148871 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:51:15.148822 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jvvrx"] Jul 08 21:51:15.154283 ip-10-0-139-229 kubenswrapper[2580]: W0708 21:51:15.154253 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dbb2389_5909_4750_99bc_8dfdc353d20e.slice/crio-9fbae1d1183d4d1765f9f0f8342e90e053d9cb97158207b722543b83afae7051 WatchSource:0}: Error finding container 9fbae1d1183d4d1765f9f0f8342e90e053d9cb97158207b722543b83afae7051: Status 404 returned error can't find the container with id 9fbae1d1183d4d1765f9f0f8342e90e053d9cb97158207b722543b83afae7051 Jul 08 21:51:16.082211 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:51:16.082178 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jvvrx" event={"ID":"2dbb2389-5909-4750-99bc-8dfdc353d20e","Type":"ContainerStarted","Data":"67f3d60ee0e1f4e9ea40b9f2d1b9662868a03c12ae6eedeafec338578df8680d"} Jul 08 21:51:16.082605 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:51:16.082242 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jvvrx" Jul 08 21:51:16.082605 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:51:16.082267 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jvvrx" event={"ID":"2dbb2389-5909-4750-99bc-8dfdc353d20e","Type":"ContainerStarted","Data":"9fbae1d1183d4d1765f9f0f8342e90e053d9cb97158207b722543b83afae7051"} Jul 08 21:51:16.104433 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:51:16.104381 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jvvrx" podStartSLOduration=2.104362674 podStartE2EDuration="2.104362674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:51:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:51:16.10323002 +0000 UTC m=+1098.401114967" watchObservedRunningTime="2026-07-08 21:51:16.104362674 +0000 UTC m=+1098.402247609" Jul 08 21:51:27.089372 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:51:27.089342 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jvvrx" Jul 08 21:52:40.892399 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:52:40.892359 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:52:54.515100 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:52:54.515064 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:52:54.615300 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:52:54.615266 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:54:18.200423 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:54:18.200327 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:56:54.708458 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:56:54.708380 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:06.359714 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:06.359673 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:16.455531 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:16.455492 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:19.800078 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:19.800037 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:22.705296 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:22.705264 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:22.802843 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:22.802785 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:23.180247 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:23.180214 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:28.952997 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:28.952964 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:29.018862 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:29.018831 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:29.169777 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:29.169748 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:29.227293 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:29.227212 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:30.259993 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:30.259952 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:30.345151 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:30.345118 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:30.425039 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:30.425008 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:30.585392 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:30.585360 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:30.912183 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:30.912105 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:31.010003 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:31.009975 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:32.703730 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:32.703698 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:35.504721 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:35.504681 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:35.618697 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:35.618664 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:35.724857 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:35.724823 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:35.826894 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:35.826866 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:37.624829 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:37.624788 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:41.737936 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:41.737903 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:41.794049 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:41.794007 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:41.943446 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:41.943408 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:42.061051 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:42.061020 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:43.137324 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:43.137292 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:43.228695 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:43.228659 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:43.300031 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:43.299994 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:43.698041 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:43.698005 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:43.797822 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:43.797778 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:43.898228 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:43.898196 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:46.228514 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:46.228480 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:48.187928 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:48.187901 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:48.500336 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:48.500232 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:54.293279 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:54.293242 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:54.395907 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:54.395871 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:57:56.471139 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:57:56.471108 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:00.129229 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:00.129192 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:00.229525 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:00.229494 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:00.334100 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:00.334059 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:00.435546 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:00.435460 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:01.435041 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:01.435010 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:01.530462 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:01.530431 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:01.632038 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:01.632005 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:02.153008 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:02.152976 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:02.287428 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:02.287383 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:02.392321 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:02.392288 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:03.563292 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:03.563265 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:05.880687 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:05.880656 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:06.062794 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:06.062761 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:06.169460 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:06.169384 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:06.238231 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:06.238204 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:07.311937 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:07.311906 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:07.412274 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:07.412243 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:07.599526 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:07.599497 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:11.919527 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:11.919497 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:12.019586 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:12.019548 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:12.457977 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:12.457939 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:12.543936 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:12.543902 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:13.947715 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:13.947682 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:19.636708 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:19.636621 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:19.703038 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:19.703007 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:19.798055 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:19.798023 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:19.894718 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:19.894638 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:20.013451 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:20.013422 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:20.397455 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:20.397426 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:46.335480 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:46.335446 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:52.030644 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:52.030610 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:52.100481 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:52.100447 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:52.190804 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:52.190749 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:58:52.283834 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:58:52.283734 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:08.149042 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:08.149001 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:13.906415 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:13.906376 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:14.009431 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:14.009395 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:14.107630 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:14.107579 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:24.584987 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:24.584948 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:25.777763 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:25.777730 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:31.136848 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:31.136792 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:31.240574 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:31.240536 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:31.337973 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:31.337940 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:31.403262 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:31.403187 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:32.166669 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:32.166629 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:32.264082 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:32.264031 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:32.366961 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:32.366927 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:33.889057 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:33.889025 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:39.654211 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:39.654170 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:39.714128 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:39.714096 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:40.317511 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:40.317479 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:46.063153 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:46.063119 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:46.128074 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:46.128022 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:46.284997 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:46.284967 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:46.600412 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:46.600376 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:46.705035 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:46.705006 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:49.418228 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:49.418143 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:49.508028 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:49.507995 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:49.602042 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:49.602006 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:49.812487 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:49.812450 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:50.079564 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:50.079460 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:50.171530 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:50.171492 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:53.882870 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:53.882837 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:54.562743 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:54.562704 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:54.735997 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:54.735968 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 21:59:54.980914 ip-10-0-139-229 kubenswrapper[2580]: I0708 21:59:54.980833 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:00.134983 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:00.134947 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725800-lkppm"] Jul 08 22:00:00.138340 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:00.138323 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725800-lkppm" Jul 08 22:00:00.141838 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:00.141789 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 22:00:00.141967 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:00.141862 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 22:00:00.142746 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:00.142730 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-m94p2\"" Jul 08 22:00:00.149703 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:00.149678 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725800-lkppm"] Jul 08 22:00:00.234138 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:00.234101 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4n7r\" (UniqueName: \"kubernetes.io/projected/4918576a-6ae7-415a-8cb1-39afd86cf745-kube-api-access-g4n7r\") pod \"maas-api-key-cleanup-29725800-lkppm\" (UID: \"4918576a-6ae7-415a-8cb1-39afd86cf745\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725800-lkppm" Jul 08 22:00:00.335122 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:00.335086 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4n7r\" (UniqueName: \"kubernetes.io/projected/4918576a-6ae7-415a-8cb1-39afd86cf745-kube-api-access-g4n7r\") pod \"maas-api-key-cleanup-29725800-lkppm\" (UID: \"4918576a-6ae7-415a-8cb1-39afd86cf745\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725800-lkppm" Jul 08 22:00:00.344607 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:00.344576 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4n7r\" (UniqueName: \"kubernetes.io/projected/4918576a-6ae7-415a-8cb1-39afd86cf745-kube-api-access-g4n7r\") pod \"maas-api-key-cleanup-29725800-lkppm\" (UID: \"4918576a-6ae7-415a-8cb1-39afd86cf745\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725800-lkppm" Jul 08 22:00:00.451717 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:00.451629 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725800-lkppm" Jul 08 22:00:00.582096 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:00.582073 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725800-lkppm"] Jul 08 22:00:00.584223 ip-10-0-139-229 kubenswrapper[2580]: W0708 22:00:00.584188 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4918576a_6ae7_415a_8cb1_39afd86cf745.slice/crio-1597a10ce73d2761a8f860d1685d191e1799160d6a17b70fc90a1b278ffea4bb WatchSource:0}: Error finding container 1597a10ce73d2761a8f860d1685d191e1799160d6a17b70fc90a1b278ffea4bb: Status 404 returned error can't find the container with id 1597a10ce73d2761a8f860d1685d191e1799160d6a17b70fc90a1b278ffea4bb Jul 08 22:00:00.586196 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:00.586179 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 22:00:00.942144 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:00.942105 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:01.081124 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:01.081088 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:01.369095 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:01.369057 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:01.422145 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:01.422111 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725800-lkppm" event={"ID":"4918576a-6ae7-415a-8cb1-39afd86cf745","Type":"ContainerStarted","Data":"1597a10ce73d2761a8f860d1685d191e1799160d6a17b70fc90a1b278ffea4bb"} Jul 08 22:00:02.018375 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:02.018342 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:02.178558 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:02.178523 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:06.447849 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:06.447799 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725800-lkppm" event={"ID":"4918576a-6ae7-415a-8cb1-39afd86cf745","Type":"ContainerStarted","Data":"4f92a4fc211eb638c7ecfde7c55b6ac4cad4c9ebb44321e663ff287f657f6034"} Jul 08 22:00:06.463374 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:06.463324 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725800-lkppm" podStartSLOduration=1.699544188 podStartE2EDuration="6.463305536s" totalImagesPullingTime="4.763761348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:00:06.462695447 +0000 UTC m=+1628.760580382" watchObservedRunningTime="2026-07-08 22:00:06.463305536 +0000 UTC m=+1628.761190470" Jul 08 22:00:07.592400 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:07.592365 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:07.655094 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:07.655062 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:07.880324 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:07.880237 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:08.204504 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:08.204428 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:08.299702 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:08.299669 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:08.398757 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:08.398724 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:13.730409 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:13.730375 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:13.830517 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:13.830475 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:14.123344 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:14.123307 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:14.239692 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:14.239655 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:14.324567 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:14.324540 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:14.485530 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:14.485449 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:20.110623 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:20.110595 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:20.469838 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:20.469713 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:26.812363 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:26.812296 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:27.122411 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:27.122319 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:27.463622 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:27.463519 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:30.284528 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:30.284485 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:30.409376 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:30.409328 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:30.498512 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:30.498479 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:31.049860 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:31.049830 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:31.159859 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:31.159805 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:34.775687 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:34.775651 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:40.183225 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.183186 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv"] Jul 08 22:00:40.187391 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.187372 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:40.190196 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.190167 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e7bf8858\"/\"openshift-service-ca.crt\"" Jul 08 22:00:40.190331 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.190230 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e7bf8858\"/\"kube-root-ca.crt\"" Jul 08 22:00:40.191192 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.191171 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e7bf8858\"/\"default-dockercfg-dzmfs\"" Jul 08 22:00:40.191319 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.191262 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e7bf8858\"/\"e2e-lifecycle-model-e7bf8858-kserve-self-signed-certs\"" Jul 08 22:00:40.192688 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.192646 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12efd8d2-2871-4640-957b-453a89405d78-tls-certs\") pod \"e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:40.192800 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.192718 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-home\") pod \"e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:40.192800 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.192747 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-tmp-dir\") pod \"e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:40.192800 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.192772 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tq29\" (UniqueName: \"kubernetes.io/projected/12efd8d2-2871-4640-957b-453a89405d78-kube-api-access-5tq29\") pod \"e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:40.193000 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.192913 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-dshm\") pod \"e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:40.193000 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.192944 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-model-cache\") pod \"e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:40.195052 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.194713 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv"] Jul 08 22:00:40.293484 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.293448 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12efd8d2-2871-4640-957b-453a89405d78-tls-certs\") pod \"e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:40.293670 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.293505 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-home\") pod \"e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:40.293670 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.293555 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-tmp-dir\") pod \"e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:40.293670 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.293594 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tq29\" (UniqueName: \"kubernetes.io/projected/12efd8d2-2871-4640-957b-453a89405d78-kube-api-access-5tq29\") pod \"e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:40.293873 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.293709 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-dshm\") pod \"e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:40.293873 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.293733 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-model-cache\") pod \"e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:40.293987 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.293874 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-home\") pod \"e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:40.294166 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.294138 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-tmp-dir\") pod \"e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:40.294305 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.294242 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-model-cache\") pod \"e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:40.296152 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.296131 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-dshm\") pod \"e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:40.296234 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.296187 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12efd8d2-2871-4640-957b-453a89405d78-tls-certs\") pod \"e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:40.302517 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.302499 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tq29\" (UniqueName: \"kubernetes.io/projected/12efd8d2-2871-4640-957b-453a89405d78-kube-api-access-5tq29\") pod \"e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:40.497730 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.497640 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:40.577756 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.577698 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:40.635913 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:40.635890 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv"] Jul 08 22:00:40.638227 ip-10-0-139-229 kubenswrapper[2580]: W0708 22:00:40.638197 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12efd8d2_2871_4640_957b_453a89405d78.slice/crio-ddb13a016e1281f05d9028ec12d870072a93f1982c3a9aac780376bab9af1ab6 WatchSource:0}: Error finding container ddb13a016e1281f05d9028ec12d870072a93f1982c3a9aac780376bab9af1ab6: Status 404 returned error can't find the container with id ddb13a016e1281f05d9028ec12d870072a93f1982c3a9aac780376bab9af1ab6 Jul 08 22:00:41.614967 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:41.614936 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" event={"ID":"12efd8d2-2871-4640-957b-453a89405d78","Type":"ContainerStarted","Data":"eb16b915ef1add15dc14774db861a780287d936d8c174bbb17d0b0e16b445fad"} Jul 08 22:00:41.615332 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:41.614975 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" event={"ID":"12efd8d2-2871-4640-957b-453a89405d78","Type":"ContainerStarted","Data":"ddb13a016e1281f05d9028ec12d870072a93f1982c3a9aac780376bab9af1ab6"} Jul 08 22:00:41.631202 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:41.631155 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" podStartSLOduration=1.631138478 podStartE2EDuration="1.631138478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:00:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:00:41.63018989 +0000 UTC m=+1663.928074825" watchObservedRunningTime="2026-07-08 22:00:41.631138478 +0000 UTC m=+1663.929023415" Jul 08 22:00:50.497739 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:50.497699 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:50.498160 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:50.497751 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:50.509981 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:50.509957 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:50.667410 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:50.667378 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:00:53.182311 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:53.182250 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:53.283438 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:53.283376 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:00:56.532160 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:00:56.532119 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:01:00.010530 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:00.010496 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725800-lkppm"] Jul 08 22:01:00.010953 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:00.010689 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725800-lkppm" podUID="4918576a-6ae7-415a-8cb1-39afd86cf745" containerName="cleanup" containerID="cri-o://4f92a4fc211eb638c7ecfde7c55b6ac4cad4c9ebb44321e663ff287f657f6034" gracePeriod=30 Jul 08 22:01:12.392626 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:12.392593 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:01:12.921273 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:12.921238 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:01:13.023682 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:13.023631 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:01:13.189898 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:13.189757 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:01:18.478400 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:18.478295 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:01:23.456156 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:23.456119 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv"] Jul 08 22:01:23.551043 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:23.551013 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:01:30.184120 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:30.184087 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725800-lkppm" Jul 08 22:01:30.240792 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:30.240761 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4n7r\" (UniqueName: \"kubernetes.io/projected/4918576a-6ae7-415a-8cb1-39afd86cf745-kube-api-access-g4n7r\") pod \"4918576a-6ae7-415a-8cb1-39afd86cf745\" (UID: \"4918576a-6ae7-415a-8cb1-39afd86cf745\") " Jul 08 22:01:30.243612 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:30.243579 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4918576a-6ae7-415a-8cb1-39afd86cf745-kube-api-access-g4n7r" (OuterVolumeSpecName: "kube-api-access-g4n7r") pod "4918576a-6ae7-415a-8cb1-39afd86cf745" (UID: "4918576a-6ae7-415a-8cb1-39afd86cf745"). InnerVolumeSpecName "kube-api-access-g4n7r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:01:30.342253 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:30.342218 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4n7r\" (UniqueName: \"kubernetes.io/projected/4918576a-6ae7-415a-8cb1-39afd86cf745-kube-api-access-g4n7r\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:01:30.843964 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:30.843924 2580 generic.go:356] "Generic (PLEG): container finished" podID="4918576a-6ae7-415a-8cb1-39afd86cf745" containerID="4f92a4fc211eb638c7ecfde7c55b6ac4cad4c9ebb44321e663ff287f657f6034" exitCode=137 Jul 08 22:01:30.844153 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:30.843972 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725800-lkppm" Jul 08 22:01:30.844153 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:30.844015 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725800-lkppm" event={"ID":"4918576a-6ae7-415a-8cb1-39afd86cf745","Type":"ContainerDied","Data":"4f92a4fc211eb638c7ecfde7c55b6ac4cad4c9ebb44321e663ff287f657f6034"} Jul 08 22:01:30.844153 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:30.844071 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725800-lkppm" event={"ID":"4918576a-6ae7-415a-8cb1-39afd86cf745","Type":"ContainerDied","Data":"1597a10ce73d2761a8f860d1685d191e1799160d6a17b70fc90a1b278ffea4bb"} Jul 08 22:01:30.844153 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:30.844092 2580 scope.go:117] "RemoveContainer" containerID="4f92a4fc211eb638c7ecfde7c55b6ac4cad4c9ebb44321e663ff287f657f6034" Jul 08 22:01:30.855220 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:30.855205 2580 scope.go:117] "RemoveContainer" containerID="4f92a4fc211eb638c7ecfde7c55b6ac4cad4c9ebb44321e663ff287f657f6034" Jul 08 22:01:30.855477 ip-10-0-139-229 kubenswrapper[2580]: E0708 22:01:30.855456 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f92a4fc211eb638c7ecfde7c55b6ac4cad4c9ebb44321e663ff287f657f6034\": container with ID starting with 4f92a4fc211eb638c7ecfde7c55b6ac4cad4c9ebb44321e663ff287f657f6034 not found: ID does not exist" containerID="4f92a4fc211eb638c7ecfde7c55b6ac4cad4c9ebb44321e663ff287f657f6034" Jul 08 22:01:30.855546 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:30.855491 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f92a4fc211eb638c7ecfde7c55b6ac4cad4c9ebb44321e663ff287f657f6034"} err="failed to get container status \"4f92a4fc211eb638c7ecfde7c55b6ac4cad4c9ebb44321e663ff287f657f6034\": rpc error: code = NotFound desc = could not find container \"4f92a4fc211eb638c7ecfde7c55b6ac4cad4c9ebb44321e663ff287f657f6034\": container with ID starting with 4f92a4fc211eb638c7ecfde7c55b6ac4cad4c9ebb44321e663ff287f657f6034 not found: ID does not exist" Jul 08 22:01:30.865005 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:30.864984 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725800-lkppm"] Jul 08 22:01:30.868712 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:30.868689 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725800-lkppm"] Jul 08 22:01:32.243942 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:32.243909 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4918576a-6ae7-415a-8cb1-39afd86cf745" path="/var/lib/kubelet/pods/4918576a-6ae7-415a-8cb1-39afd86cf745/volumes" Jul 08 22:01:38.476902 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.476850 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" podUID="12efd8d2-2871-4640-957b-453a89405d78" containerName="main" containerID="cri-o://eb16b915ef1add15dc14774db861a780287d936d8c174bbb17d0b0e16b445fad" gracePeriod=45 Jul 08 22:01:38.729597 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.729535 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:01:38.816240 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.816203 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-model-cache\") pod \"12efd8d2-2871-4640-957b-453a89405d78\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " Jul 08 22:01:38.816240 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.816249 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-tmp-dir\") pod \"12efd8d2-2871-4640-957b-453a89405d78\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " Jul 08 22:01:38.816470 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.816287 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-dshm\") pod \"12efd8d2-2871-4640-957b-453a89405d78\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " Jul 08 22:01:38.816470 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.816329 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tq29\" (UniqueName: \"kubernetes.io/projected/12efd8d2-2871-4640-957b-453a89405d78-kube-api-access-5tq29\") pod \"12efd8d2-2871-4640-957b-453a89405d78\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " Jul 08 22:01:38.816470 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.816370 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12efd8d2-2871-4640-957b-453a89405d78-tls-certs\") pod \"12efd8d2-2871-4640-957b-453a89405d78\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " Jul 08 22:01:38.816470 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.816416 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-home\") pod \"12efd8d2-2871-4640-957b-453a89405d78\" (UID: \"12efd8d2-2871-4640-957b-453a89405d78\") " Jul 08 22:01:38.816669 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.816514 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-model-cache" (OuterVolumeSpecName: "model-cache") pod "12efd8d2-2871-4640-957b-453a89405d78" (UID: "12efd8d2-2871-4640-957b-453a89405d78"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:01:38.816669 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.816605 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "12efd8d2-2871-4640-957b-453a89405d78" (UID: "12efd8d2-2871-4640-957b-453a89405d78"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:01:38.816669 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.816641 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-model-cache\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:01:38.816793 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.816736 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-home" (OuterVolumeSpecName: "home") pod "12efd8d2-2871-4640-957b-453a89405d78" (UID: "12efd8d2-2871-4640-957b-453a89405d78"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:01:38.818640 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.818618 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-dshm" (OuterVolumeSpecName: "dshm") pod "12efd8d2-2871-4640-957b-453a89405d78" (UID: "12efd8d2-2871-4640-957b-453a89405d78"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:01:38.818726 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.818708 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12efd8d2-2871-4640-957b-453a89405d78-kube-api-access-5tq29" (OuterVolumeSpecName: "kube-api-access-5tq29") pod "12efd8d2-2871-4640-957b-453a89405d78" (UID: "12efd8d2-2871-4640-957b-453a89405d78"). InnerVolumeSpecName "kube-api-access-5tq29". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:01:38.818764 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.818724 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12efd8d2-2871-4640-957b-453a89405d78-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "12efd8d2-2871-4640-957b-453a89405d78" (UID: "12efd8d2-2871-4640-957b-453a89405d78"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:01:38.889782 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.889748 2580 generic.go:356] "Generic (PLEG): container finished" podID="12efd8d2-2871-4640-957b-453a89405d78" containerID="eb16b915ef1add15dc14774db861a780287d936d8c174bbb17d0b0e16b445fad" exitCode=0 Jul 08 22:01:38.889919 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.889790 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" Jul 08 22:01:38.889919 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.889839 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" event={"ID":"12efd8d2-2871-4640-957b-453a89405d78","Type":"ContainerDied","Data":"eb16b915ef1add15dc14774db861a780287d936d8c174bbb17d0b0e16b445fad"} Jul 08 22:01:38.889919 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.889883 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv" event={"ID":"12efd8d2-2871-4640-957b-453a89405d78","Type":"ContainerDied","Data":"ddb13a016e1281f05d9028ec12d870072a93f1982c3a9aac780376bab9af1ab6"} Jul 08 22:01:38.889919 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.889902 2580 scope.go:117] "RemoveContainer" containerID="eb16b915ef1add15dc14774db861a780287d936d8c174bbb17d0b0e16b445fad" Jul 08 22:01:38.899539 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.899522 2580 scope.go:117] "RemoveContainer" containerID="eb16b915ef1add15dc14774db861a780287d936d8c174bbb17d0b0e16b445fad" Jul 08 22:01:38.899775 ip-10-0-139-229 kubenswrapper[2580]: E0708 22:01:38.899759 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb16b915ef1add15dc14774db861a780287d936d8c174bbb17d0b0e16b445fad\": container with ID starting with eb16b915ef1add15dc14774db861a780287d936d8c174bbb17d0b0e16b445fad not found: ID does not exist" containerID="eb16b915ef1add15dc14774db861a780287d936d8c174bbb17d0b0e16b445fad" Jul 08 22:01:38.899832 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.899785 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb16b915ef1add15dc14774db861a780287d936d8c174bbb17d0b0e16b445fad"} err="failed to get container status \"eb16b915ef1add15dc14774db861a780287d936d8c174bbb17d0b0e16b445fad\": rpc error: code = NotFound desc = could not find container \"eb16b915ef1add15dc14774db861a780287d936d8c174bbb17d0b0e16b445fad\": container with ID starting with eb16b915ef1add15dc14774db861a780287d936d8c174bbb17d0b0e16b445fad not found: ID does not exist" Jul 08 22:01:38.912663 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.912637 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv"] Jul 08 22:01:38.917754 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.917736 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-home\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:01:38.917754 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.917754 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-tmp-dir\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:01:38.917886 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.917762 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/12efd8d2-2871-4640-957b-453a89405d78-dshm\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:01:38.917886 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.917770 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tq29\" (UniqueName: \"kubernetes.io/projected/12efd8d2-2871-4640-957b-453a89405d78-kube-api-access-5tq29\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:01:38.917886 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.917779 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/12efd8d2-2871-4640-957b-453a89405d78-tls-certs\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:01:38.918151 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:38.918134 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-e7bf8858/e2e-lifecycle-model-e7bf8858-kserve-d54969659-6spsv"] Jul 08 22:01:40.242888 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:40.242853 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12efd8d2-2871-4640-957b-453a89405d78" path="/var/lib/kubelet/pods/12efd8d2-2871-4640-957b-453a89405d78/volumes" Jul 08 22:01:59.979576 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:01:59.979536 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:00.699464 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:00.699427 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:06.156979 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:06.156943 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:06.226407 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:06.226372 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:06.476882 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:06.476787 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:07.785997 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:07.785962 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:07.884686 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:07.884643 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:07.988398 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:07.988366 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:08.762388 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:08.762351 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:08.856495 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:08.856460 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:08.957594 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:08.957564 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:10.418750 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:10.418719 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:16.529436 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:16.529402 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:16.631880 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:16.631844 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:17.239389 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:17.239329 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:23.743071 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:23.743028 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:23.842469 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:23.842424 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:24.038701 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:24.038661 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:24.153072 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:24.153017 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:34.712028 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:34.711989 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:35.056521 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:35.056485 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:02:56.029677 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:02:56.029641 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:01.718643 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:01.718609 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:01.782308 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:01.782279 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:02.090108 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:02.090076 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:02.429089 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:02.429011 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:02.620351 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:02.620320 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:05.133018 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:05.132987 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:05.206029 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:05.205999 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:05.275973 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:05.275941 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:05.495560 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:05.495474 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:05.805753 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:05.805718 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:05.907310 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:05.907278 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:09.680592 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:09.680548 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:10.005549 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:10.005405 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:10.312408 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:10.312375 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:10.637293 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:10.637200 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:13.172232 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.172197 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:13.270425 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.270387 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:13.380870 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.380838 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh"] Jul 08 22:03:13.381204 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.381189 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4918576a-6ae7-415a-8cb1-39afd86cf745" containerName="cleanup" Jul 08 22:03:13.381204 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.381203 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4918576a-6ae7-415a-8cb1-39afd86cf745" containerName="cleanup" Jul 08 22:03:13.381348 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.381217 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12efd8d2-2871-4640-957b-453a89405d78" containerName="main" Jul 08 22:03:13.381348 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.381223 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="12efd8d2-2871-4640-957b-453a89405d78" containerName="main" Jul 08 22:03:13.381348 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.381284 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="12efd8d2-2871-4640-957b-453a89405d78" containerName="main" Jul 08 22:03:13.381348 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.381291 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4918576a-6ae7-415a-8cb1-39afd86cf745" containerName="cleanup" Jul 08 22:03:13.384215 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.384192 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" Jul 08 22:03:13.386878 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.386855 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-b6d4ea-serving-cert\"" Jul 08 22:03:13.386981 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.386898 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 22:03:13.386981 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.386919 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-m94p2\"" Jul 08 22:03:13.387180 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.387161 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 22:03:13.394862 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.394840 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh"] Jul 08 22:03:13.443798 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.443727 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:13.453903 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.453879 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5t67\" (UniqueName: \"kubernetes.io/projected/1c9c773b-64d3-4c08-a203-ef7d97cbc202-kube-api-access-b5t67\") pod \"maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh\" (UID: \"1c9c773b-64d3-4c08-a203-ef7d97cbc202\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" Jul 08 22:03:13.454017 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.453928 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c9c773b-64d3-4c08-a203-ef7d97cbc202-maas-api-tls\") pod \"maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh\" (UID: \"1c9c773b-64d3-4c08-a203-ef7d97cbc202\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" Jul 08 22:03:13.454017 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.453946 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c9c773b-64d3-4c08-a203-ef7d97cbc202-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh\" (UID: \"1c9c773b-64d3-4c08-a203-ef7d97cbc202\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" Jul 08 22:03:13.554490 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.554457 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5t67\" (UniqueName: \"kubernetes.io/projected/1c9c773b-64d3-4c08-a203-ef7d97cbc202-kube-api-access-b5t67\") pod \"maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh\" (UID: \"1c9c773b-64d3-4c08-a203-ef7d97cbc202\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" Jul 08 22:03:13.554671 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.554550 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c9c773b-64d3-4c08-a203-ef7d97cbc202-maas-api-tls\") pod \"maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh\" (UID: \"1c9c773b-64d3-4c08-a203-ef7d97cbc202\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" Jul 08 22:03:13.554671 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.554575 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c9c773b-64d3-4c08-a203-ef7d97cbc202-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh\" (UID: \"1c9c773b-64d3-4c08-a203-ef7d97cbc202\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" Jul 08 22:03:13.555265 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.555241 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c9c773b-64d3-4c08-a203-ef7d97cbc202-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh\" (UID: \"1c9c773b-64d3-4c08-a203-ef7d97cbc202\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" Jul 08 22:03:13.557190 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.557166 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c9c773b-64d3-4c08-a203-ef7d97cbc202-maas-api-tls\") pod \"maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh\" (UID: \"1c9c773b-64d3-4c08-a203-ef7d97cbc202\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" Jul 08 22:03:13.562827 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.562790 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5t67\" (UniqueName: \"kubernetes.io/projected/1c9c773b-64d3-4c08-a203-ef7d97cbc202-kube-api-access-b5t67\") pod \"maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh\" (UID: \"1c9c773b-64d3-4c08-a203-ef7d97cbc202\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" Jul 08 22:03:13.639215 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.639177 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:13.694850 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.694753 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" Jul 08 22:03:13.841677 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:13.841652 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh"] Jul 08 22:03:13.844280 ip-10-0-139-229 kubenswrapper[2580]: W0708 22:03:13.844258 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c9c773b_64d3_4c08_a203_ef7d97cbc202.slice/crio-66a3904b5e214dc5e735ffbbafd67e8fec0108ae268b3d465ca4f2be086b6d07 WatchSource:0}: Error finding container 66a3904b5e214dc5e735ffbbafd67e8fec0108ae268b3d465ca4f2be086b6d07: Status 404 returned error can't find the container with id 66a3904b5e214dc5e735ffbbafd67e8fec0108ae268b3d465ca4f2be086b6d07 Jul 08 22:03:14.066758 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:14.066723 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:14.162587 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:14.162557 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:14.331134 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:14.331052 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" event={"ID":"1c9c773b-64d3-4c08-a203-ef7d97cbc202","Type":"ContainerStarted","Data":"66a3904b5e214dc5e735ffbbafd67e8fec0108ae268b3d465ca4f2be086b6d07"} Jul 08 22:03:16.342309 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:16.342280 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" event={"ID":"1c9c773b-64d3-4c08-a203-ef7d97cbc202","Type":"ContainerStarted","Data":"6a5021ed8e3656a4cd56fed2697b58d6aba52819db773818ddd27d0d7f1cfe6f"} Jul 08 22:03:16.342689 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:16.342398 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" Jul 08 22:03:16.359742 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:16.359693 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" podStartSLOduration=1.670981338 podStartE2EDuration="3.359678783s" totalImagesPullingTime="1.688697445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:03:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:03:16.35879237 +0000 UTC m=+1818.656677304" watchObservedRunningTime="2026-07-08 22:03:16.359678783 +0000 UTC m=+1818.657563717" Jul 08 22:03:17.619703 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:17.619669 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:22.352627 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:22.352592 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" Jul 08 22:03:27.595346 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:27.595308 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:27.695402 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:27.695366 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:30.571043 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.571016 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb"] Jul 08 22:03:30.574638 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.574617 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:30.578140 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.578115 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-0303e4\"/\"openshift-service-ca.crt\"" Jul 08 22:03:30.578256 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.578149 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-0303e4\"/\"default-dockercfg-spq8b\"" Jul 08 22:03:30.578256 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.578171 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-0303e4\"/\"kube-root-ca.crt\"" Jul 08 22:03:30.578256 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.578209 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-0303e4\"/\"test-model-0303e4-kserve-self-signed-certs\"" Jul 08 22:03:30.583607 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.583587 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb"] Jul 08 22:03:30.591030 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.591003 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:30.707882 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.707848 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-model-cache\") pod \"test-model-0303e4-kserve-fc578f5ff-dfbxb\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:30.708036 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.707904 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-dshm\") pod \"test-model-0303e4-kserve-fc578f5ff-dfbxb\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:30.708036 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.707928 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-home\") pod \"test-model-0303e4-kserve-fc578f5ff-dfbxb\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:30.708036 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.707945 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8jjb\" (UniqueName: \"kubernetes.io/projected/8e0453db-e234-4373-8486-2d12191e31de-kube-api-access-w8jjb\") pod \"test-model-0303e4-kserve-fc578f5ff-dfbxb\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:30.708036 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.708016 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-tmp-dir\") pod \"test-model-0303e4-kserve-fc578f5ff-dfbxb\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:30.708169 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.708046 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e0453db-e234-4373-8486-2d12191e31de-tls-certs\") pod \"test-model-0303e4-kserve-fc578f5ff-dfbxb\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:30.809162 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.809133 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-tmp-dir\") pod \"test-model-0303e4-kserve-fc578f5ff-dfbxb\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:30.809317 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.809170 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e0453db-e234-4373-8486-2d12191e31de-tls-certs\") pod \"test-model-0303e4-kserve-fc578f5ff-dfbxb\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:30.809317 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.809217 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-model-cache\") pod \"test-model-0303e4-kserve-fc578f5ff-dfbxb\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:30.809317 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.809262 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-dshm\") pod \"test-model-0303e4-kserve-fc578f5ff-dfbxb\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:30.809484 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.809462 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-home\") pod \"test-model-0303e4-kserve-fc578f5ff-dfbxb\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:30.809539 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.809525 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8jjb\" (UniqueName: \"kubernetes.io/projected/8e0453db-e234-4373-8486-2d12191e31de-kube-api-access-w8jjb\") pod \"test-model-0303e4-kserve-fc578f5ff-dfbxb\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:30.809670 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.809643 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-tmp-dir\") pod \"test-model-0303e4-kserve-fc578f5ff-dfbxb\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:30.809845 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.809678 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-model-cache\") pod \"test-model-0303e4-kserve-fc578f5ff-dfbxb\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:30.809935 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.809863 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-home\") pod \"test-model-0303e4-kserve-fc578f5ff-dfbxb\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:30.811745 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.811725 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-dshm\") pod \"test-model-0303e4-kserve-fc578f5ff-dfbxb\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:30.811845 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.811787 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e0453db-e234-4373-8486-2d12191e31de-tls-certs\") pod \"test-model-0303e4-kserve-fc578f5ff-dfbxb\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:30.817299 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.817275 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8jjb\" (UniqueName: \"kubernetes.io/projected/8e0453db-e234-4373-8486-2d12191e31de-kube-api-access-w8jjb\") pod \"test-model-0303e4-kserve-fc578f5ff-dfbxb\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:30.883731 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:30.883668 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:31.017404 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:31.017378 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb"] Jul 08 22:03:31.019605 ip-10-0-139-229 kubenswrapper[2580]: W0708 22:03:31.019580 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e0453db_e234_4373_8486_2d12191e31de.slice/crio-8a6b872858194cb42e36ccaa2d17ecce736844cf9fdabe204dc2a3498c7a6f78 WatchSource:0}: Error finding container 8a6b872858194cb42e36ccaa2d17ecce736844cf9fdabe204dc2a3498c7a6f78: Status 404 returned error can't find the container with id 8a6b872858194cb42e36ccaa2d17ecce736844cf9fdabe204dc2a3498c7a6f78 Jul 08 22:03:31.414859 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:31.414799 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" event={"ID":"8e0453db-e234-4373-8486-2d12191e31de","Type":"ContainerStarted","Data":"ec6d8b0e9ac5424736c5aa16b9b39bb7f43f8e99092d13e6f9740b88c62e6936"} Jul 08 22:03:31.415025 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:31.414871 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" event={"ID":"8e0453db-e234-4373-8486-2d12191e31de","Type":"ContainerStarted","Data":"8a6b872858194cb42e36ccaa2d17ecce736844cf9fdabe204dc2a3498c7a6f78"} Jul 08 22:03:31.434523 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:31.434477 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" podStartSLOduration=1.434459556 podStartE2EDuration="1.434459556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:03:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:03:31.434219372 +0000 UTC m=+1833.732104320" watchObservedRunningTime="2026-07-08 22:03:31.434459556 +0000 UTC m=+1833.732344492" Jul 08 22:03:35.644089 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:35.644043 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:35.744061 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:35.744027 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:40.883975 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:40.883931 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:40.883975 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:40.883979 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:40.896281 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:40.896255 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:41.471848 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:41.471822 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:03:47.691037 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.691000 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4"] Jul 08 22:03:47.694517 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.694500 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:47.697979 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.697958 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-b6d4ea\"/\"default-dockercfg-nvw66\"" Jul 08 22:03:47.697979 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.697976 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-b6d4ea\"/\"openshift-service-ca.crt\"" Jul 08 22:03:47.698142 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.697987 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-b6d4ea\"/\"kube-root-ca.crt\"" Jul 08 22:03:47.698142 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.698001 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-b6d4ea\"/\"test-model-b6d4ea-kserve-self-signed-certs\"" Jul 08 22:03:47.704861 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.704840 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4"] Jul 08 22:03:47.718000 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.717971 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:03:47.763483 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.763455 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-dshm\") pod \"test-model-b6d4ea-kserve-65948b99c7-mmpm4\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:47.763589 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.763493 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-model-cache\") pod \"test-model-b6d4ea-kserve-65948b99c7-mmpm4\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:47.763589 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.763516 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-home\") pod \"test-model-b6d4ea-kserve-65948b99c7-mmpm4\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:47.763589 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.763539 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d6xc\" (UniqueName: \"kubernetes.io/projected/fc32927d-e770-4007-b58f-abe4b6c6e857-kube-api-access-6d6xc\") pod \"test-model-b6d4ea-kserve-65948b99c7-mmpm4\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:47.763754 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.763588 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc32927d-e770-4007-b58f-abe4b6c6e857-tls-certs\") pod \"test-model-b6d4ea-kserve-65948b99c7-mmpm4\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:47.763754 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.763635 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-tmp-dir\") pod \"test-model-b6d4ea-kserve-65948b99c7-mmpm4\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:47.864346 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.864308 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-dshm\") pod \"test-model-b6d4ea-kserve-65948b99c7-mmpm4\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:47.864511 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.864361 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-model-cache\") pod \"test-model-b6d4ea-kserve-65948b99c7-mmpm4\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:47.864511 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.864390 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-home\") pod \"test-model-b6d4ea-kserve-65948b99c7-mmpm4\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:47.864511 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.864420 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d6xc\" (UniqueName: \"kubernetes.io/projected/fc32927d-e770-4007-b58f-abe4b6c6e857-kube-api-access-6d6xc\") pod \"test-model-b6d4ea-kserve-65948b99c7-mmpm4\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:47.864511 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.864444 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc32927d-e770-4007-b58f-abe4b6c6e857-tls-certs\") pod \"test-model-b6d4ea-kserve-65948b99c7-mmpm4\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:47.864511 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.864473 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-tmp-dir\") pod \"test-model-b6d4ea-kserve-65948b99c7-mmpm4\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:47.864879 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.864860 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-home\") pod \"test-model-b6d4ea-kserve-65948b99c7-mmpm4\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:47.864960 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.864923 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-tmp-dir\") pod \"test-model-b6d4ea-kserve-65948b99c7-mmpm4\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:47.864960 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.864951 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-model-cache\") pod \"test-model-b6d4ea-kserve-65948b99c7-mmpm4\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:47.866763 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.866746 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-dshm\") pod \"test-model-b6d4ea-kserve-65948b99c7-mmpm4\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:47.866959 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.866944 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc32927d-e770-4007-b58f-abe4b6c6e857-tls-certs\") pod \"test-model-b6d4ea-kserve-65948b99c7-mmpm4\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:47.884335 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:47.884305 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d6xc\" (UniqueName: \"kubernetes.io/projected/fc32927d-e770-4007-b58f-abe4b6c6e857-kube-api-access-6d6xc\") pod \"test-model-b6d4ea-kserve-65948b99c7-mmpm4\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:48.005021 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:48.004935 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:48.136751 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:48.136605 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4"] Jul 08 22:03:48.138769 ip-10-0-139-229 kubenswrapper[2580]: W0708 22:03:48.138742 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc32927d_e770_4007_b58f_abe4b6c6e857.slice/crio-b6f8bcf72d504a84cadd22c49d653fefce876f595aa47b2039c85d247aa24584 WatchSource:0}: Error finding container b6f8bcf72d504a84cadd22c49d653fefce876f595aa47b2039c85d247aa24584: Status 404 returned error can't find the container with id b6f8bcf72d504a84cadd22c49d653fefce876f595aa47b2039c85d247aa24584 Jul 08 22:03:48.492234 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:48.492195 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" event={"ID":"fc32927d-e770-4007-b58f-abe4b6c6e857","Type":"ContainerStarted","Data":"5c506e076a849dd4f269d32c583dc571182d22da263f8fa0eaf032e6d192c86e"} Jul 08 22:03:48.492395 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:48.492242 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" event={"ID":"fc32927d-e770-4007-b58f-abe4b6c6e857","Type":"ContainerStarted","Data":"b6f8bcf72d504a84cadd22c49d653fefce876f595aa47b2039c85d247aa24584"} Jul 08 22:03:48.517739 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:48.517680 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" podStartSLOduration=1.517665359 podStartE2EDuration="1.517665359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:03:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:03:48.515897702 +0000 UTC m=+1850.813782637" watchObservedRunningTime="2026-07-08 22:03:48.517665359 +0000 UTC m=+1850.815550294" Jul 08 22:03:58.005491 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:58.005455 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:58.005966 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:58.005504 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:58.018009 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:58.017985 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:03:58.549763 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:03:58.549732 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:04:05.822960 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:05.822929 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb"] Jul 08 22:04:05.844521 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:05.844490 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:04:06.731326 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:06.731285 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4"] Jul 08 22:04:06.752889 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:06.752858 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:04:06.923905 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:06.923873 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:04:07.327193 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:07.327163 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:04:07.427169 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:07.427133 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:04:07.532823 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:07.532788 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:04:07.634156 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:07.634079 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:04:12.396795 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:12.396765 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:04:20.845172 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:20.845091 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" podUID="8e0453db-e234-4373-8486-2d12191e31de" containerName="main" containerID="cri-o://ec6d8b0e9ac5424736c5aa16b9b39bb7f43f8e99092d13e6f9740b88c62e6936" gracePeriod=45 Jul 08 22:04:21.101944 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.101890 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:04:21.264792 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.264761 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-home\") pod \"8e0453db-e234-4373-8486-2d12191e31de\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " Jul 08 22:04:21.264980 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.264825 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-tmp-dir\") pod \"8e0453db-e234-4373-8486-2d12191e31de\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " Jul 08 22:04:21.264980 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.264852 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-model-cache\") pod \"8e0453db-e234-4373-8486-2d12191e31de\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " Jul 08 22:04:21.264980 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.264879 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-dshm\") pod \"8e0453db-e234-4373-8486-2d12191e31de\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " Jul 08 22:04:21.264980 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.264900 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8jjb\" (UniqueName: \"kubernetes.io/projected/8e0453db-e234-4373-8486-2d12191e31de-kube-api-access-w8jjb\") pod \"8e0453db-e234-4373-8486-2d12191e31de\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " Jul 08 22:04:21.264980 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.264930 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e0453db-e234-4373-8486-2d12191e31de-tls-certs\") pod \"8e0453db-e234-4373-8486-2d12191e31de\" (UID: \"8e0453db-e234-4373-8486-2d12191e31de\") " Jul 08 22:04:21.265247 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.264992 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-home" (OuterVolumeSpecName: "home") pod "8e0453db-e234-4373-8486-2d12191e31de" (UID: "8e0453db-e234-4373-8486-2d12191e31de"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:04:21.265247 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.265141 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-model-cache" (OuterVolumeSpecName: "model-cache") pod "8e0453db-e234-4373-8486-2d12191e31de" (UID: "8e0453db-e234-4373-8486-2d12191e31de"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:04:21.265247 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.265153 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8e0453db-e234-4373-8486-2d12191e31de" (UID: "8e0453db-e234-4373-8486-2d12191e31de"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:04:21.265394 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.265273 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-home\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:04:21.265394 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.265311 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-tmp-dir\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:04:21.265394 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.265323 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-model-cache\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:04:21.267237 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.267212 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e0453db-e234-4373-8486-2d12191e31de-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8e0453db-e234-4373-8486-2d12191e31de" (UID: "8e0453db-e234-4373-8486-2d12191e31de"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:04:21.267354 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.267241 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-dshm" (OuterVolumeSpecName: "dshm") pod "8e0453db-e234-4373-8486-2d12191e31de" (UID: "8e0453db-e234-4373-8486-2d12191e31de"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:04:21.267354 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.267337 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e0453db-e234-4373-8486-2d12191e31de-kube-api-access-w8jjb" (OuterVolumeSpecName: "kube-api-access-w8jjb") pod "8e0453db-e234-4373-8486-2d12191e31de" (UID: "8e0453db-e234-4373-8486-2d12191e31de"). InnerVolumeSpecName "kube-api-access-w8jjb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:04:21.366579 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.366504 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e0453db-e234-4373-8486-2d12191e31de-dshm\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:04:21.366579 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.366529 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8jjb\" (UniqueName: \"kubernetes.io/projected/8e0453db-e234-4373-8486-2d12191e31de-kube-api-access-w8jjb\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:04:21.366579 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.366544 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e0453db-e234-4373-8486-2d12191e31de-tls-certs\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:04:21.662609 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.662528 2580 generic.go:356] "Generic (PLEG): container finished" podID="8e0453db-e234-4373-8486-2d12191e31de" containerID="ec6d8b0e9ac5424736c5aa16b9b39bb7f43f8e99092d13e6f9740b88c62e6936" exitCode=0 Jul 08 22:04:21.662609 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.662571 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" Jul 08 22:04:21.662846 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.662609 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" event={"ID":"8e0453db-e234-4373-8486-2d12191e31de","Type":"ContainerDied","Data":"ec6d8b0e9ac5424736c5aa16b9b39bb7f43f8e99092d13e6f9740b88c62e6936"} Jul 08 22:04:21.662846 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.662655 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb" event={"ID":"8e0453db-e234-4373-8486-2d12191e31de","Type":"ContainerDied","Data":"8a6b872858194cb42e36ccaa2d17ecce736844cf9fdabe204dc2a3498c7a6f78"} Jul 08 22:04:21.662846 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.662677 2580 scope.go:117] "RemoveContainer" containerID="ec6d8b0e9ac5424736c5aa16b9b39bb7f43f8e99092d13e6f9740b88c62e6936" Jul 08 22:04:21.671700 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.671686 2580 scope.go:117] "RemoveContainer" containerID="ec6d8b0e9ac5424736c5aa16b9b39bb7f43f8e99092d13e6f9740b88c62e6936" Jul 08 22:04:21.671954 ip-10-0-139-229 kubenswrapper[2580]: E0708 22:04:21.671936 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec6d8b0e9ac5424736c5aa16b9b39bb7f43f8e99092d13e6f9740b88c62e6936\": container with ID starting with ec6d8b0e9ac5424736c5aa16b9b39bb7f43f8e99092d13e6f9740b88c62e6936 not found: ID does not exist" containerID="ec6d8b0e9ac5424736c5aa16b9b39bb7f43f8e99092d13e6f9740b88c62e6936" Jul 08 22:04:21.672007 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.671962 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec6d8b0e9ac5424736c5aa16b9b39bb7f43f8e99092d13e6f9740b88c62e6936"} err="failed to get container status \"ec6d8b0e9ac5424736c5aa16b9b39bb7f43f8e99092d13e6f9740b88c62e6936\": rpc error: code = NotFound desc = could not find container \"ec6d8b0e9ac5424736c5aa16b9b39bb7f43f8e99092d13e6f9740b88c62e6936\": container with ID starting with ec6d8b0e9ac5424736c5aa16b9b39bb7f43f8e99092d13e6f9740b88c62e6936 not found: ID does not exist" Jul 08 22:04:21.688041 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.688020 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb"] Jul 08 22:04:21.699109 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.699085 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-0303e4/test-model-0303e4-kserve-fc578f5ff-dfbxb"] Jul 08 22:04:21.751931 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.751896 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" podUID="fc32927d-e770-4007-b58f-abe4b6c6e857" containerName="main" containerID="cri-o://5c506e076a849dd4f269d32c583dc571182d22da263f8fa0eaf032e6d192c86e" gracePeriod=45 Jul 08 22:04:21.996991 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:21.996967 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:04:22.172525 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.172447 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-dshm\") pod \"fc32927d-e770-4007-b58f-abe4b6c6e857\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " Jul 08 22:04:22.172525 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.172499 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-tmp-dir\") pod \"fc32927d-e770-4007-b58f-abe4b6c6e857\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " Jul 08 22:04:22.172525 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.172528 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6d6xc\" (UniqueName: \"kubernetes.io/projected/fc32927d-e770-4007-b58f-abe4b6c6e857-kube-api-access-6d6xc\") pod \"fc32927d-e770-4007-b58f-abe4b6c6e857\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " Jul 08 22:04:22.172801 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.172555 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-model-cache\") pod \"fc32927d-e770-4007-b58f-abe4b6c6e857\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " Jul 08 22:04:22.172801 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.172577 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-home\") pod \"fc32927d-e770-4007-b58f-abe4b6c6e857\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " Jul 08 22:04:22.172801 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.172642 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc32927d-e770-4007-b58f-abe4b6c6e857-tls-certs\") pod \"fc32927d-e770-4007-b58f-abe4b6c6e857\" (UID: \"fc32927d-e770-4007-b58f-abe4b6c6e857\") " Jul 08 22:04:22.172993 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.172867 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-model-cache" (OuterVolumeSpecName: "model-cache") pod "fc32927d-e770-4007-b58f-abe4b6c6e857" (UID: "fc32927d-e770-4007-b58f-abe4b6c6e857"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:04:22.172993 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.172882 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fc32927d-e770-4007-b58f-abe4b6c6e857" (UID: "fc32927d-e770-4007-b58f-abe4b6c6e857"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:04:22.172993 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.172930 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-home" (OuterVolumeSpecName: "home") pod "fc32927d-e770-4007-b58f-abe4b6c6e857" (UID: "fc32927d-e770-4007-b58f-abe4b6c6e857"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:04:22.174865 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.174845 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-dshm" (OuterVolumeSpecName: "dshm") pod "fc32927d-e770-4007-b58f-abe4b6c6e857" (UID: "fc32927d-e770-4007-b58f-abe4b6c6e857"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 22:04:22.174950 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.174914 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc32927d-e770-4007-b58f-abe4b6c6e857-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fc32927d-e770-4007-b58f-abe4b6c6e857" (UID: "fc32927d-e770-4007-b58f-abe4b6c6e857"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:04:22.174989 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.174959 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc32927d-e770-4007-b58f-abe4b6c6e857-kube-api-access-6d6xc" (OuterVolumeSpecName: "kube-api-access-6d6xc") pod "fc32927d-e770-4007-b58f-abe4b6c6e857" (UID: "fc32927d-e770-4007-b58f-abe4b6c6e857"). InnerVolumeSpecName "kube-api-access-6d6xc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:04:22.243039 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.243014 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e0453db-e234-4373-8486-2d12191e31de" path="/var/lib/kubelet/pods/8e0453db-e234-4373-8486-2d12191e31de/volumes" Jul 08 22:04:22.274139 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.274113 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-model-cache\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:04:22.274139 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.274136 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-home\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:04:22.274265 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.274144 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc32927d-e770-4007-b58f-abe4b6c6e857-tls-certs\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:04:22.274265 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.274151 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-dshm\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:04:22.274265 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.274158 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc32927d-e770-4007-b58f-abe4b6c6e857-tmp-dir\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:04:22.274265 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.274165 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6d6xc\" (UniqueName: \"kubernetes.io/projected/fc32927d-e770-4007-b58f-abe4b6c6e857-kube-api-access-6d6xc\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:04:22.668880 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.668849 2580 generic.go:356] "Generic (PLEG): container finished" podID="fc32927d-e770-4007-b58f-abe4b6c6e857" containerID="5c506e076a849dd4f269d32c583dc571182d22da263f8fa0eaf032e6d192c86e" exitCode=0 Jul 08 22:04:22.669049 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.668901 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" Jul 08 22:04:22.669049 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.668939 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" event={"ID":"fc32927d-e770-4007-b58f-abe4b6c6e857","Type":"ContainerDied","Data":"5c506e076a849dd4f269d32c583dc571182d22da263f8fa0eaf032e6d192c86e"} Jul 08 22:04:22.669049 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.668977 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4" event={"ID":"fc32927d-e770-4007-b58f-abe4b6c6e857","Type":"ContainerDied","Data":"b6f8bcf72d504a84cadd22c49d653fefce876f595aa47b2039c85d247aa24584"} Jul 08 22:04:22.669049 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.668993 2580 scope.go:117] "RemoveContainer" containerID="5c506e076a849dd4f269d32c583dc571182d22da263f8fa0eaf032e6d192c86e" Jul 08 22:04:22.677672 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.677633 2580 scope.go:117] "RemoveContainer" containerID="5c506e076a849dd4f269d32c583dc571182d22da263f8fa0eaf032e6d192c86e" Jul 08 22:04:22.677937 ip-10-0-139-229 kubenswrapper[2580]: E0708 22:04:22.677916 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c506e076a849dd4f269d32c583dc571182d22da263f8fa0eaf032e6d192c86e\": container with ID starting with 5c506e076a849dd4f269d32c583dc571182d22da263f8fa0eaf032e6d192c86e not found: ID does not exist" containerID="5c506e076a849dd4f269d32c583dc571182d22da263f8fa0eaf032e6d192c86e" Jul 08 22:04:22.678026 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.677943 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c506e076a849dd4f269d32c583dc571182d22da263f8fa0eaf032e6d192c86e"} err="failed to get container status \"5c506e076a849dd4f269d32c583dc571182d22da263f8fa0eaf032e6d192c86e\": rpc error: code = NotFound desc = could not find container \"5c506e076a849dd4f269d32c583dc571182d22da263f8fa0eaf032e6d192c86e\": container with ID starting with 5c506e076a849dd4f269d32c583dc571182d22da263f8fa0eaf032e6d192c86e not found: ID does not exist" Jul 08 22:04:22.684613 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.684591 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4"] Jul 08 22:04:22.689481 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:22.689461 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-b6d4ea/test-model-b6d4ea-kserve-65948b99c7-mmpm4"] Jul 08 22:04:24.242471 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:24.242438 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc32927d-e770-4007-b58f-abe4b6c6e857" path="/var/lib/kubelet/pods/fc32927d-e770-4007-b58f-abe4b6c6e857/volumes" Jul 08 22:04:49.119593 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:49.119555 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:04:50.801667 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:50.801632 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh"] Jul 08 22:04:50.802143 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:50.801863 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" podUID="1c9c773b-64d3-4c08-a203-ef7d97cbc202" containerName="maas-api" containerID="cri-o://6a5021ed8e3656a4cd56fed2697b58d6aba52819db773818ddd27d0d7f1cfe6f" gracePeriod=30 Jul 08 22:04:51.062416 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:51.062359 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" Jul 08 22:04:51.072962 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:51.072929 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:04:51.121787 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:51.121754 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c9c773b-64d3-4c08-a203-ef7d97cbc202-maas-api-tls\") pod \"1c9c773b-64d3-4c08-a203-ef7d97cbc202\" (UID: \"1c9c773b-64d3-4c08-a203-ef7d97cbc202\") " Jul 08 22:04:51.121949 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:51.121854 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c9c773b-64d3-4c08-a203-ef7d97cbc202-openshift-service-ca\") pod \"1c9c773b-64d3-4c08-a203-ef7d97cbc202\" (UID: \"1c9c773b-64d3-4c08-a203-ef7d97cbc202\") " Jul 08 22:04:51.121949 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:51.121907 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5t67\" (UniqueName: \"kubernetes.io/projected/1c9c773b-64d3-4c08-a203-ef7d97cbc202-kube-api-access-b5t67\") pod \"1c9c773b-64d3-4c08-a203-ef7d97cbc202\" (UID: \"1c9c773b-64d3-4c08-a203-ef7d97cbc202\") " Jul 08 22:04:51.122226 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:51.122195 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c9c773b-64d3-4c08-a203-ef7d97cbc202-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1c9c773b-64d3-4c08-a203-ef7d97cbc202" (UID: "1c9c773b-64d3-4c08-a203-ef7d97cbc202"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:04:51.124074 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:51.124052 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c9c773b-64d3-4c08-a203-ef7d97cbc202-kube-api-access-b5t67" (OuterVolumeSpecName: "kube-api-access-b5t67") pod "1c9c773b-64d3-4c08-a203-ef7d97cbc202" (UID: "1c9c773b-64d3-4c08-a203-ef7d97cbc202"). InnerVolumeSpecName "kube-api-access-b5t67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:04:51.124159 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:51.124078 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c9c773b-64d3-4c08-a203-ef7d97cbc202-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1c9c773b-64d3-4c08-a203-ef7d97cbc202" (UID: "1c9c773b-64d3-4c08-a203-ef7d97cbc202"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:04:51.222687 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:51.222655 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c9c773b-64d3-4c08-a203-ef7d97cbc202-maas-api-tls\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:04:51.222687 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:51.222683 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c9c773b-64d3-4c08-a203-ef7d97cbc202-openshift-service-ca\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:04:51.222687 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:51.222692 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5t67\" (UniqueName: \"kubernetes.io/projected/1c9c773b-64d3-4c08-a203-ef7d97cbc202-kube-api-access-b5t67\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:04:51.801251 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:51.801219 2580 generic.go:356] "Generic (PLEG): container finished" podID="1c9c773b-64d3-4c08-a203-ef7d97cbc202" containerID="6a5021ed8e3656a4cd56fed2697b58d6aba52819db773818ddd27d0d7f1cfe6f" exitCode=0 Jul 08 22:04:51.801421 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:51.801273 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" Jul 08 22:04:51.801421 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:51.801300 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" event={"ID":"1c9c773b-64d3-4c08-a203-ef7d97cbc202","Type":"ContainerDied","Data":"6a5021ed8e3656a4cd56fed2697b58d6aba52819db773818ddd27d0d7f1cfe6f"} Jul 08 22:04:51.801421 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:51.801350 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh" event={"ID":"1c9c773b-64d3-4c08-a203-ef7d97cbc202","Type":"ContainerDied","Data":"66a3904b5e214dc5e735ffbbafd67e8fec0108ae268b3d465ca4f2be086b6d07"} Jul 08 22:04:51.801421 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:51.801372 2580 scope.go:117] "RemoveContainer" containerID="6a5021ed8e3656a4cd56fed2697b58d6aba52819db773818ddd27d0d7f1cfe6f" Jul 08 22:04:51.810160 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:51.810145 2580 scope.go:117] "RemoveContainer" containerID="6a5021ed8e3656a4cd56fed2697b58d6aba52819db773818ddd27d0d7f1cfe6f" Jul 08 22:04:51.810450 ip-10-0-139-229 kubenswrapper[2580]: E0708 22:04:51.810425 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a5021ed8e3656a4cd56fed2697b58d6aba52819db773818ddd27d0d7f1cfe6f\": container with ID starting with 6a5021ed8e3656a4cd56fed2697b58d6aba52819db773818ddd27d0d7f1cfe6f not found: ID does not exist" containerID="6a5021ed8e3656a4cd56fed2697b58d6aba52819db773818ddd27d0d7f1cfe6f" Jul 08 22:04:51.810491 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:51.810465 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a5021ed8e3656a4cd56fed2697b58d6aba52819db773818ddd27d0d7f1cfe6f"} err="failed to get container status \"6a5021ed8e3656a4cd56fed2697b58d6aba52819db773818ddd27d0d7f1cfe6f\": rpc error: code = NotFound desc = could not find container \"6a5021ed8e3656a4cd56fed2697b58d6aba52819db773818ddd27d0d7f1cfe6f\": container with ID starting with 6a5021ed8e3656a4cd56fed2697b58d6aba52819db773818ddd27d0d7f1cfe6f not found: ID does not exist" Jul 08 22:04:51.821923 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:51.821894 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh"] Jul 08 22:04:51.831776 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:51.831749 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b6d4ea-f66dfb6f7-bzrvh"] Jul 08 22:04:52.242943 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:52.242861 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c9c773b-64d3-4c08-a203-ef7d97cbc202" path="/var/lib/kubelet/pods/1c9c773b-64d3-4c08-a203-ef7d97cbc202/volumes" Jul 08 22:04:55.857257 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:55.857223 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:04:55.959536 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:55.959492 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:04:56.097688 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:56.097660 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:04:56.678559 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:56.678525 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:04:56.994389 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:56.994309 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:04:57.188912 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:57.188879 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:04:59.800078 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.800044 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982"] Jul 08 22:04:59.800457 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.800385 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e0453db-e234-4373-8486-2d12191e31de" containerName="main" Jul 08 22:04:59.800457 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.800396 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e0453db-e234-4373-8486-2d12191e31de" containerName="main" Jul 08 22:04:59.800457 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.800407 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c9c773b-64d3-4c08-a203-ef7d97cbc202" containerName="maas-api" Jul 08 22:04:59.800457 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.800412 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c9c773b-64d3-4c08-a203-ef7d97cbc202" containerName="maas-api" Jul 08 22:04:59.800457 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.800429 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc32927d-e770-4007-b58f-abe4b6c6e857" containerName="main" Jul 08 22:04:59.800457 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.800434 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc32927d-e770-4007-b58f-abe4b6c6e857" containerName="main" Jul 08 22:04:59.800668 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.800485 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc32927d-e770-4007-b58f-abe4b6c6e857" containerName="main" Jul 08 22:04:59.800668 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.800494 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c9c773b-64d3-4c08-a203-ef7d97cbc202" containerName="maas-api" Jul 08 22:04:59.800668 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.800501 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e0453db-e234-4373-8486-2d12191e31de" containerName="main" Jul 08 22:04:59.803310 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.803292 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" Jul 08 22:04:59.805986 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.805963 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-c32bb0-serving-cert\"" Jul 08 22:04:59.806102 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.806036 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 22:04:59.806212 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.806196 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 22:04:59.807029 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.807010 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-m94p2\"" Jul 08 22:04:59.812653 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.812633 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982"] Jul 08 22:04:59.877324 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.877287 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:04:59.906493 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.906462 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/793e36fa-9cd2-4fcc-96c2-c9705788c1a7-openshift-service-ca\") pod \"maas-api-e2e-api-a-c32bb0-6958b47b9-9m982\" (UID: \"793e36fa-9cd2-4fcc-96c2-c9705788c1a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" Jul 08 22:04:59.906672 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.906533 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/793e36fa-9cd2-4fcc-96c2-c9705788c1a7-maas-api-tls\") pod \"maas-api-e2e-api-a-c32bb0-6958b47b9-9m982\" (UID: \"793e36fa-9cd2-4fcc-96c2-c9705788c1a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" Jul 08 22:04:59.906672 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.906576 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fsgj\" (UniqueName: \"kubernetes.io/projected/793e36fa-9cd2-4fcc-96c2-c9705788c1a7-kube-api-access-7fsgj\") pod \"maas-api-e2e-api-a-c32bb0-6958b47b9-9m982\" (UID: \"793e36fa-9cd2-4fcc-96c2-c9705788c1a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" Jul 08 22:04:59.977634 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:04:59.977597 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:00.007913 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:00.007883 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/793e36fa-9cd2-4fcc-96c2-c9705788c1a7-maas-api-tls\") pod \"maas-api-e2e-api-a-c32bb0-6958b47b9-9m982\" (UID: \"793e36fa-9cd2-4fcc-96c2-c9705788c1a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" Jul 08 22:05:00.008062 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:00.007942 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fsgj\" (UniqueName: \"kubernetes.io/projected/793e36fa-9cd2-4fcc-96c2-c9705788c1a7-kube-api-access-7fsgj\") pod \"maas-api-e2e-api-a-c32bb0-6958b47b9-9m982\" (UID: \"793e36fa-9cd2-4fcc-96c2-c9705788c1a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" Jul 08 22:05:00.008062 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:00.007980 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/793e36fa-9cd2-4fcc-96c2-c9705788c1a7-openshift-service-ca\") pod \"maas-api-e2e-api-a-c32bb0-6958b47b9-9m982\" (UID: \"793e36fa-9cd2-4fcc-96c2-c9705788c1a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" Jul 08 22:05:00.008644 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:00.008578 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/793e36fa-9cd2-4fcc-96c2-c9705788c1a7-openshift-service-ca\") pod \"maas-api-e2e-api-a-c32bb0-6958b47b9-9m982\" (UID: \"793e36fa-9cd2-4fcc-96c2-c9705788c1a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" Jul 08 22:05:00.010425 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:00.010397 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/793e36fa-9cd2-4fcc-96c2-c9705788c1a7-maas-api-tls\") pod \"maas-api-e2e-api-a-c32bb0-6958b47b9-9m982\" (UID: \"793e36fa-9cd2-4fcc-96c2-c9705788c1a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" Jul 08 22:05:00.017271 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:00.017246 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fsgj\" (UniqueName: \"kubernetes.io/projected/793e36fa-9cd2-4fcc-96c2-c9705788c1a7-kube-api-access-7fsgj\") pod \"maas-api-e2e-api-a-c32bb0-6958b47b9-9m982\" (UID: \"793e36fa-9cd2-4fcc-96c2-c9705788c1a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" Jul 08 22:05:00.078249 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:00.078143 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:00.114369 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:00.114333 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" Jul 08 22:05:00.277609 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:00.277586 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982"] Jul 08 22:05:00.280477 ip-10-0-139-229 kubenswrapper[2580]: W0708 22:05:00.280437 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793e36fa_9cd2_4fcc_96c2_c9705788c1a7.slice/crio-329faa6a00c64d3eefd2e23c509b8a15ee0121eaa1226a55e095d22bdc1dd658 WatchSource:0}: Error finding container 329faa6a00c64d3eefd2e23c509b8a15ee0121eaa1226a55e095d22bdc1dd658: Status 404 returned error can't find the container with id 329faa6a00c64d3eefd2e23c509b8a15ee0121eaa1226a55e095d22bdc1dd658 Jul 08 22:05:00.383786 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:00.383706 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:00.472343 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:00.472315 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:00.843915 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:00.843874 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" event={"ID":"793e36fa-9cd2-4fcc-96c2-c9705788c1a7","Type":"ContainerStarted","Data":"fe8f5077c82fb71647306d92f8cc45845874865782537f8efd0b3a38dce76a59"} Jul 08 22:05:00.844379 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:00.843940 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" event={"ID":"793e36fa-9cd2-4fcc-96c2-c9705788c1a7","Type":"ContainerStarted","Data":"329faa6a00c64d3eefd2e23c509b8a15ee0121eaa1226a55e095d22bdc1dd658"} Jul 08 22:05:00.844379 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:00.844032 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" Jul 08 22:05:00.861857 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:00.861788 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" podStartSLOduration=1.469645522 podStartE2EDuration="1.861772277s" totalImagesPullingTime="392.126755ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:05:00.860479005 +0000 UTC m=+1923.158363940" watchObservedRunningTime="2026-07-08 22:05:00.861772277 +0000 UTC m=+1923.159657212" Jul 08 22:05:04.229229 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:04.229200 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:04.672926 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:04.672892 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:05.105784 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:05.105749 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:05.206828 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:05.206790 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:06.854951 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:06.854920 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" Jul 08 22:05:07.765148 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:07.765109 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:07.869644 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:07.869616 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:07.882041 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:07.882011 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp"] Jul 08 22:05:07.886072 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:07.886052 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" Jul 08 22:05:07.888660 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:07.888641 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-dc88f4-serving-cert\"" Jul 08 22:05:07.901463 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:07.900532 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp"] Jul 08 22:05:07.963483 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:07.963450 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:07.978648 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:07.978621 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecc287dc-c48b-4a20-a2ba-f9a0d229e619-maas-api-tls\") pod \"maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp\" (UID: \"ecc287dc-c48b-4a20-a2ba-f9a0d229e619\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" Jul 08 22:05:07.978776 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:07.978653 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpw5z\" (UniqueName: \"kubernetes.io/projected/ecc287dc-c48b-4a20-a2ba-f9a0d229e619-kube-api-access-cpw5z\") pod \"maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp\" (UID: \"ecc287dc-c48b-4a20-a2ba-f9a0d229e619\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" Jul 08 22:05:07.978776 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:07.978692 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecc287dc-c48b-4a20-a2ba-f9a0d229e619-openshift-service-ca\") pod \"maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp\" (UID: \"ecc287dc-c48b-4a20-a2ba-f9a0d229e619\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" Jul 08 22:05:08.079947 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:08.079910 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecc287dc-c48b-4a20-a2ba-f9a0d229e619-maas-api-tls\") pod \"maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp\" (UID: \"ecc287dc-c48b-4a20-a2ba-f9a0d229e619\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" Jul 08 22:05:08.080095 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:08.079954 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpw5z\" (UniqueName: \"kubernetes.io/projected/ecc287dc-c48b-4a20-a2ba-f9a0d229e619-kube-api-access-cpw5z\") pod \"maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp\" (UID: \"ecc287dc-c48b-4a20-a2ba-f9a0d229e619\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" Jul 08 22:05:08.080095 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:08.080005 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecc287dc-c48b-4a20-a2ba-f9a0d229e619-openshift-service-ca\") pod \"maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp\" (UID: \"ecc287dc-c48b-4a20-a2ba-f9a0d229e619\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" Jul 08 22:05:08.080807 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:08.080741 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecc287dc-c48b-4a20-a2ba-f9a0d229e619-openshift-service-ca\") pod \"maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp\" (UID: \"ecc287dc-c48b-4a20-a2ba-f9a0d229e619\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" Jul 08 22:05:08.082564 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:08.082543 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecc287dc-c48b-4a20-a2ba-f9a0d229e619-maas-api-tls\") pod \"maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp\" (UID: \"ecc287dc-c48b-4a20-a2ba-f9a0d229e619\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" Jul 08 22:05:08.090396 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:08.090375 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpw5z\" (UniqueName: \"kubernetes.io/projected/ecc287dc-c48b-4a20-a2ba-f9a0d229e619-kube-api-access-cpw5z\") pod \"maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp\" (UID: \"ecc287dc-c48b-4a20-a2ba-f9a0d229e619\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" Jul 08 22:05:08.136028 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:08.136000 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:08.197366 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:08.197338 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" Jul 08 22:05:08.350086 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:08.350008 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp"] Jul 08 22:05:08.353036 ip-10-0-139-229 kubenswrapper[2580]: W0708 22:05:08.353012 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecc287dc_c48b_4a20_a2ba_f9a0d229e619.slice/crio-008c434b652044f9586b02265032ad0aeca4d72209cbbaedc1ba200312daa605 WatchSource:0}: Error finding container 008c434b652044f9586b02265032ad0aeca4d72209cbbaedc1ba200312daa605: Status 404 returned error can't find the container with id 008c434b652044f9586b02265032ad0aeca4d72209cbbaedc1ba200312daa605 Jul 08 22:05:08.354314 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:08.354294 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 22:05:08.567161 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:08.567125 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:08.694541 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:08.694454 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:08.883533 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:08.883498 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" event={"ID":"ecc287dc-c48b-4a20-a2ba-f9a0d229e619","Type":"ContainerStarted","Data":"4c198663b0b363c7efb95b03667bacd9a2a14c46fb616a9b89c4f32e1eac0a78"} Jul 08 22:05:08.883934 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:08.883553 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" event={"ID":"ecc287dc-c48b-4a20-a2ba-f9a0d229e619","Type":"ContainerStarted","Data":"008c434b652044f9586b02265032ad0aeca4d72209cbbaedc1ba200312daa605"} Jul 08 22:05:08.883934 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:08.883585 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" Jul 08 22:05:08.901034 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:08.900987 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" podStartSLOduration=1.523188057 podStartE2EDuration="1.900971682s" totalImagesPullingTime="377.783625ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:05:08.900101955 +0000 UTC m=+1931.197986893" watchObservedRunningTime="2026-07-08 22:05:08.900971682 +0000 UTC m=+1931.198856616" Jul 08 22:05:12.234669 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:12.234635 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:14.893711 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:14.893685 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" Jul 08 22:05:20.585340 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:20.585291 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:20.965269 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:20.965188 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:21.082764 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.082731 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:21.186028 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.185986 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:21.330078 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.330046 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982"] Jul 08 22:05:21.330283 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.330245 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" podUID="793e36fa-9cd2-4fcc-96c2-c9705788c1a7" containerName="maas-api" containerID="cri-o://fe8f5077c82fb71647306d92f8cc45845874865782537f8efd0b3a38dce76a59" gracePeriod=30 Jul 08 22:05:21.342796 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.342767 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:21.574880 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.574857 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" Jul 08 22:05:21.600972 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.600915 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/793e36fa-9cd2-4fcc-96c2-c9705788c1a7-maas-api-tls\") pod \"793e36fa-9cd2-4fcc-96c2-c9705788c1a7\" (UID: \"793e36fa-9cd2-4fcc-96c2-c9705788c1a7\") " Jul 08 22:05:21.600972 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.600946 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fsgj\" (UniqueName: \"kubernetes.io/projected/793e36fa-9cd2-4fcc-96c2-c9705788c1a7-kube-api-access-7fsgj\") pod \"793e36fa-9cd2-4fcc-96c2-c9705788c1a7\" (UID: \"793e36fa-9cd2-4fcc-96c2-c9705788c1a7\") " Jul 08 22:05:21.601325 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.601003 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/793e36fa-9cd2-4fcc-96c2-c9705788c1a7-openshift-service-ca\") pod \"793e36fa-9cd2-4fcc-96c2-c9705788c1a7\" (UID: \"793e36fa-9cd2-4fcc-96c2-c9705788c1a7\") " Jul 08 22:05:21.601448 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.601417 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/793e36fa-9cd2-4fcc-96c2-c9705788c1a7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "793e36fa-9cd2-4fcc-96c2-c9705788c1a7" (UID: "793e36fa-9cd2-4fcc-96c2-c9705788c1a7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:05:21.603177 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.603153 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/793e36fa-9cd2-4fcc-96c2-c9705788c1a7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "793e36fa-9cd2-4fcc-96c2-c9705788c1a7" (UID: "793e36fa-9cd2-4fcc-96c2-c9705788c1a7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:05:21.603261 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.603171 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/793e36fa-9cd2-4fcc-96c2-c9705788c1a7-kube-api-access-7fsgj" (OuterVolumeSpecName: "kube-api-access-7fsgj") pod "793e36fa-9cd2-4fcc-96c2-c9705788c1a7" (UID: "793e36fa-9cd2-4fcc-96c2-c9705788c1a7"). InnerVolumeSpecName "kube-api-access-7fsgj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:05:21.701787 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.701742 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/793e36fa-9cd2-4fcc-96c2-c9705788c1a7-maas-api-tls\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:05:21.701787 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.701782 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fsgj\" (UniqueName: \"kubernetes.io/projected/793e36fa-9cd2-4fcc-96c2-c9705788c1a7-kube-api-access-7fsgj\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:05:21.701787 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.701792 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/793e36fa-9cd2-4fcc-96c2-c9705788c1a7-openshift-service-ca\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:05:21.942434 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.942357 2580 generic.go:356] "Generic (PLEG): container finished" podID="793e36fa-9cd2-4fcc-96c2-c9705788c1a7" containerID="fe8f5077c82fb71647306d92f8cc45845874865782537f8efd0b3a38dce76a59" exitCode=0 Jul 08 22:05:21.942434 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.942414 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" Jul 08 22:05:21.942611 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.942438 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" event={"ID":"793e36fa-9cd2-4fcc-96c2-c9705788c1a7","Type":"ContainerDied","Data":"fe8f5077c82fb71647306d92f8cc45845874865782537f8efd0b3a38dce76a59"} Jul 08 22:05:21.942611 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.942490 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982" event={"ID":"793e36fa-9cd2-4fcc-96c2-c9705788c1a7","Type":"ContainerDied","Data":"329faa6a00c64d3eefd2e23c509b8a15ee0121eaa1226a55e095d22bdc1dd658"} Jul 08 22:05:21.942611 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.942511 2580 scope.go:117] "RemoveContainer" containerID="fe8f5077c82fb71647306d92f8cc45845874865782537f8efd0b3a38dce76a59" Jul 08 22:05:21.951034 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.951017 2580 scope.go:117] "RemoveContainer" containerID="fe8f5077c82fb71647306d92f8cc45845874865782537f8efd0b3a38dce76a59" Jul 08 22:05:21.951319 ip-10-0-139-229 kubenswrapper[2580]: E0708 22:05:21.951296 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe8f5077c82fb71647306d92f8cc45845874865782537f8efd0b3a38dce76a59\": container with ID starting with fe8f5077c82fb71647306d92f8cc45845874865782537f8efd0b3a38dce76a59 not found: ID does not exist" containerID="fe8f5077c82fb71647306d92f8cc45845874865782537f8efd0b3a38dce76a59" Jul 08 22:05:21.951399 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.951329 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe8f5077c82fb71647306d92f8cc45845874865782537f8efd0b3a38dce76a59"} err="failed to get container status \"fe8f5077c82fb71647306d92f8cc45845874865782537f8efd0b3a38dce76a59\": rpc error: code = NotFound desc = could not find container \"fe8f5077c82fb71647306d92f8cc45845874865782537f8efd0b3a38dce76a59\": container with ID starting with fe8f5077c82fb71647306d92f8cc45845874865782537f8efd0b3a38dce76a59 not found: ID does not exist" Jul 08 22:05:21.965384 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.965362 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982"] Jul 08 22:05:21.970342 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:21.970321 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c32bb0-6958b47b9-9m982"] Jul 08 22:05:22.243356 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:22.243278 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="793e36fa-9cd2-4fcc-96c2-c9705788c1a7" path="/var/lib/kubelet/pods/793e36fa-9cd2-4fcc-96c2-c9705788c1a7/volumes" Jul 08 22:05:26.753967 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:26.753930 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:26.851904 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:26.851871 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:27.175474 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:27.175439 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:27.358878 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:27.358849 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:27.753946 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:27.753920 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:27.855887 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:27.855857 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:27.959839 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:27.959792 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:28.019707 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.019615 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp"] Jul 08 22:05:28.019909 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.019861 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" podUID="ecc287dc-c48b-4a20-a2ba-f9a0d229e619" containerName="maas-api" containerID="cri-o://4c198663b0b363c7efb95b03667bacd9a2a14c46fb616a9b89c4f32e1eac0a78" gracePeriod=30 Jul 08 22:05:28.051852 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.051824 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:28.273115 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.273094 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" Jul 08 22:05:28.363733 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.363704 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecc287dc-c48b-4a20-a2ba-f9a0d229e619-openshift-service-ca\") pod \"ecc287dc-c48b-4a20-a2ba-f9a0d229e619\" (UID: \"ecc287dc-c48b-4a20-a2ba-f9a0d229e619\") " Jul 08 22:05:28.363894 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.363787 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpw5z\" (UniqueName: \"kubernetes.io/projected/ecc287dc-c48b-4a20-a2ba-f9a0d229e619-kube-api-access-cpw5z\") pod \"ecc287dc-c48b-4a20-a2ba-f9a0d229e619\" (UID: \"ecc287dc-c48b-4a20-a2ba-f9a0d229e619\") " Jul 08 22:05:28.363894 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.363862 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecc287dc-c48b-4a20-a2ba-f9a0d229e619-maas-api-tls\") pod \"ecc287dc-c48b-4a20-a2ba-f9a0d229e619\" (UID: \"ecc287dc-c48b-4a20-a2ba-f9a0d229e619\") " Jul 08 22:05:28.364131 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.364102 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecc287dc-c48b-4a20-a2ba-f9a0d229e619-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ecc287dc-c48b-4a20-a2ba-f9a0d229e619" (UID: "ecc287dc-c48b-4a20-a2ba-f9a0d229e619"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:05:28.364242 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.364219 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecc287dc-c48b-4a20-a2ba-f9a0d229e619-openshift-service-ca\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:05:28.366067 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.366040 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecc287dc-c48b-4a20-a2ba-f9a0d229e619-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ecc287dc-c48b-4a20-a2ba-f9a0d229e619" (UID: "ecc287dc-c48b-4a20-a2ba-f9a0d229e619"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:05:28.366163 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.366145 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecc287dc-c48b-4a20-a2ba-f9a0d229e619-kube-api-access-cpw5z" (OuterVolumeSpecName: "kube-api-access-cpw5z") pod "ecc287dc-c48b-4a20-a2ba-f9a0d229e619" (UID: "ecc287dc-c48b-4a20-a2ba-f9a0d229e619"). InnerVolumeSpecName "kube-api-access-cpw5z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:05:28.465505 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.465472 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpw5z\" (UniqueName: \"kubernetes.io/projected/ecc287dc-c48b-4a20-a2ba-f9a0d229e619-kube-api-access-cpw5z\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:05:28.465505 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.465500 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ecc287dc-c48b-4a20-a2ba-f9a0d229e619-maas-api-tls\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:05:28.976689 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.976655 2580 generic.go:356] "Generic (PLEG): container finished" podID="ecc287dc-c48b-4a20-a2ba-f9a0d229e619" containerID="4c198663b0b363c7efb95b03667bacd9a2a14c46fb616a9b89c4f32e1eac0a78" exitCode=0 Jul 08 22:05:28.977105 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.976701 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" Jul 08 22:05:28.977105 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.976733 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" event={"ID":"ecc287dc-c48b-4a20-a2ba-f9a0d229e619","Type":"ContainerDied","Data":"4c198663b0b363c7efb95b03667bacd9a2a14c46fb616a9b89c4f32e1eac0a78"} Jul 08 22:05:28.977105 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.976775 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp" event={"ID":"ecc287dc-c48b-4a20-a2ba-f9a0d229e619","Type":"ContainerDied","Data":"008c434b652044f9586b02265032ad0aeca4d72209cbbaedc1ba200312daa605"} Jul 08 22:05:28.977105 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.976795 2580 scope.go:117] "RemoveContainer" containerID="4c198663b0b363c7efb95b03667bacd9a2a14c46fb616a9b89c4f32e1eac0a78" Jul 08 22:05:28.985395 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.985375 2580 scope.go:117] "RemoveContainer" containerID="4c198663b0b363c7efb95b03667bacd9a2a14c46fb616a9b89c4f32e1eac0a78" Jul 08 22:05:28.985641 ip-10-0-139-229 kubenswrapper[2580]: E0708 22:05:28.985623 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c198663b0b363c7efb95b03667bacd9a2a14c46fb616a9b89c4f32e1eac0a78\": container with ID starting with 4c198663b0b363c7efb95b03667bacd9a2a14c46fb616a9b89c4f32e1eac0a78 not found: ID does not exist" containerID="4c198663b0b363c7efb95b03667bacd9a2a14c46fb616a9b89c4f32e1eac0a78" Jul 08 22:05:28.985690 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.985648 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c198663b0b363c7efb95b03667bacd9a2a14c46fb616a9b89c4f32e1eac0a78"} err="failed to get container status \"4c198663b0b363c7efb95b03667bacd9a2a14c46fb616a9b89c4f32e1eac0a78\": rpc error: code = NotFound desc = could not find container \"4c198663b0b363c7efb95b03667bacd9a2a14c46fb616a9b89c4f32e1eac0a78\": container with ID starting with 4c198663b0b363c7efb95b03667bacd9a2a14c46fb616a9b89c4f32e1eac0a78 not found: ID does not exist" Jul 08 22:05:28.997904 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:28.997873 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp"] Jul 08 22:05:29.002598 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:29.002578 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dc88f4-69f846df89-xgxtp"] Jul 08 22:05:30.243179 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:30.243137 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecc287dc-c48b-4a20-a2ba-f9a0d229e619" path="/var/lib/kubelet/pods/ecc287dc-c48b-4a20-a2ba-f9a0d229e619/volumes" Jul 08 22:05:33.505653 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:33.505614 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:33.606277 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:33.606245 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:33.786285 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:33.786259 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:34.386878 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:34.386842 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:34.653280 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:34.653201 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:34.870867 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:34.870806 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:37.945780 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:37.945745 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:38.034864 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.034832 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m"] Jul 08 22:05:38.035286 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.035268 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="793e36fa-9cd2-4fcc-96c2-c9705788c1a7" containerName="maas-api" Jul 08 22:05:38.035374 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.035289 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="793e36fa-9cd2-4fcc-96c2-c9705788c1a7" containerName="maas-api" Jul 08 22:05:38.035374 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.035300 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecc287dc-c48b-4a20-a2ba-f9a0d229e619" containerName="maas-api" Jul 08 22:05:38.035374 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.035309 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecc287dc-c48b-4a20-a2ba-f9a0d229e619" containerName="maas-api" Jul 08 22:05:38.035535 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.035381 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecc287dc-c48b-4a20-a2ba-f9a0d229e619" containerName="maas-api" Jul 08 22:05:38.035535 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.035399 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="793e36fa-9cd2-4fcc-96c2-c9705788c1a7" containerName="maas-api" Jul 08 22:05:38.038552 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.038533 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" Jul 08 22:05:38.040862 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.040837 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:38.041330 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.041312 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-074c17-serving-cert\"" Jul 08 22:05:38.042067 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.042049 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 22:05:38.042391 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.042376 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 22:05:38.042474 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.042413 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-m94p2\"" Jul 08 22:05:38.051785 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.051762 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m"] Jul 08 22:05:38.120004 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.119978 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:38.146183 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.146161 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m\" (UID: \"0e0e1bee-3b43-409e-a490-69fb3fb13d3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" Jul 08 22:05:38.146283 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.146196 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-maas-api-tls\") pod \"maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m\" (UID: \"0e0e1bee-3b43-409e-a490-69fb3fb13d3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" Jul 08 22:05:38.146320 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.146306 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97b82\" (UniqueName: \"kubernetes.io/projected/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-kube-api-access-97b82\") pod \"maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m\" (UID: \"0e0e1bee-3b43-409e-a490-69fb3fb13d3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" Jul 08 22:05:38.247158 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.247077 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m\" (UID: \"0e0e1bee-3b43-409e-a490-69fb3fb13d3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" Jul 08 22:05:38.247158 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.247122 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-maas-api-tls\") pod \"maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m\" (UID: \"0e0e1bee-3b43-409e-a490-69fb3fb13d3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" Jul 08 22:05:38.247356 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.247229 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97b82\" (UniqueName: \"kubernetes.io/projected/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-kube-api-access-97b82\") pod \"maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m\" (UID: \"0e0e1bee-3b43-409e-a490-69fb3fb13d3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" Jul 08 22:05:38.247356 ip-10-0-139-229 kubenswrapper[2580]: E0708 22:05:38.247283 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-serving-cert: secret "maas-api-e2e-shared-a-074c17-serving-cert" not found Jul 08 22:05:38.247356 ip-10-0-139-229 kubenswrapper[2580]: E0708 22:05:38.247341 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-maas-api-tls podName:0e0e1bee-3b43-409e-a490-69fb3fb13d3b nodeName:}" failed. No retries permitted until 2026-07-08 22:05:38.747326046 +0000 UTC m=+1961.045210959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-maas-api-tls") pod "maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" (UID: "0e0e1bee-3b43-409e-a490-69fb3fb13d3b") : secret "maas-api-e2e-shared-a-074c17-serving-cert" not found Jul 08 22:05:38.247769 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.247750 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m\" (UID: \"0e0e1bee-3b43-409e-a490-69fb3fb13d3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" Jul 08 22:05:38.257351 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.257329 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97b82\" (UniqueName: \"kubernetes.io/projected/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-kube-api-access-97b82\") pod \"maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m\" (UID: \"0e0e1bee-3b43-409e-a490-69fb3fb13d3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" Jul 08 22:05:38.492069 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.492034 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:38.634077 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.634043 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:38.732181 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.732150 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:38.750443 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.750414 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-maas-api-tls\") pod \"maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m\" (UID: \"0e0e1bee-3b43-409e-a490-69fb3fb13d3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" Jul 08 22:05:38.753016 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.752993 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-maas-api-tls\") pod \"maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m\" (UID: \"0e0e1bee-3b43-409e-a490-69fb3fb13d3b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" Jul 08 22:05:38.951176 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:38.951095 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" Jul 08 22:05:39.091545 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:39.091520 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m"] Jul 08 22:05:39.094290 ip-10-0-139-229 kubenswrapper[2580]: W0708 22:05:39.094266 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e0e1bee_3b43_409e_a490_69fb3fb13d3b.slice/crio-3b0843f483d0ee02ca8ec3c17a46c585e1f29d389f63e7fe38d99be6ad9934df WatchSource:0}: Error finding container 3b0843f483d0ee02ca8ec3c17a46c585e1f29d389f63e7fe38d99be6ad9934df: Status 404 returned error can't find the container with id 3b0843f483d0ee02ca8ec3c17a46c585e1f29d389f63e7fe38d99be6ad9934df Jul 08 22:05:40.029483 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:40.029444 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" event={"ID":"0e0e1bee-3b43-409e-a490-69fb3fb13d3b","Type":"ContainerStarted","Data":"8d7637abb8383f5283d5f36f9a4bee4d3c04df64caa1e39759ce6bd8e64f4bdc"} Jul 08 22:05:40.029857 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:40.029509 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" event={"ID":"0e0e1bee-3b43-409e-a490-69fb3fb13d3b","Type":"ContainerStarted","Data":"3b0843f483d0ee02ca8ec3c17a46c585e1f29d389f63e7fe38d99be6ad9934df"} Jul 08 22:05:40.029857 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:40.029535 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" Jul 08 22:05:40.047868 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:40.047802 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" podStartSLOduration=1.606893388 podStartE2EDuration="2.047785364s" totalImagesPullingTime="440.891976ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 22:05:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:05:40.046631779 +0000 UTC m=+1962.344516711" watchObservedRunningTime="2026-07-08 22:05:40.047785364 +0000 UTC m=+1962.345670298" Jul 08 22:05:42.481702 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:42.481670 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:42.535892 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:42.535862 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:42.870770 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:42.870735 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:43.342237 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:43.342192 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:43.686487 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:43.686407 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:46.003194 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:46.003153 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:46.039374 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:46.039345 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" Jul 08 22:05:46.098079 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:46.098046 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:46.172028 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:46.171994 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:46.451823 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:46.451770 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:46.570140 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:46.570104 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:46.670937 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:46.670902 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:50.503886 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:50.503776 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:53.929850 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:53.929804 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:54.032554 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:54.032513 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:59.334055 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:59.334007 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:05:59.435386 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:05:59.435358 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:06:00.274486 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:06:00.274456 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:06:07.291189 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:06:07.291160 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:06:07.391850 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:06:07.391796 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:06:42.334204 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:06:42.334153 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:07:12.892443 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:07:12.892401 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:07:16.577727 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:07:16.577694 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:07:16.678427 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:07:16.678360 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:08:55.696093 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:55.696020 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:08:56.010236 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.010143 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:08:56.109051 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.109018 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:08:56.250138 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.250110 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m"] Jul 08 22:08:56.250346 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.250326 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" podUID="0e0e1bee-3b43-409e-a490-69fb3fb13d3b" containerName="maas-api" containerID="cri-o://8d7637abb8383f5283d5f36f9a4bee4d3c04df64caa1e39759ce6bd8e64f4bdc" gracePeriod=30 Jul 08 22:08:56.270735 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.270712 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:08:56.497457 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.497435 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" Jul 08 22:08:56.584874 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.584831 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97b82\" (UniqueName: \"kubernetes.io/projected/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-kube-api-access-97b82\") pod \"0e0e1bee-3b43-409e-a490-69fb3fb13d3b\" (UID: \"0e0e1bee-3b43-409e-a490-69fb3fb13d3b\") " Jul 08 22:08:56.585047 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.584938 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-maas-api-tls\") pod \"0e0e1bee-3b43-409e-a490-69fb3fb13d3b\" (UID: \"0e0e1bee-3b43-409e-a490-69fb3fb13d3b\") " Jul 08 22:08:56.585047 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.584962 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-openshift-service-ca\") pod \"0e0e1bee-3b43-409e-a490-69fb3fb13d3b\" (UID: \"0e0e1bee-3b43-409e-a490-69fb3fb13d3b\") " Jul 08 22:08:56.585378 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.585358 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0e0e1bee-3b43-409e-a490-69fb3fb13d3b" (UID: "0e0e1bee-3b43-409e-a490-69fb3fb13d3b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 22:08:56.587127 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.587110 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0e0e1bee-3b43-409e-a490-69fb3fb13d3b" (UID: "0e0e1bee-3b43-409e-a490-69fb3fb13d3b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 22:08:56.587201 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.587125 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-kube-api-access-97b82" (OuterVolumeSpecName: "kube-api-access-97b82") pod "0e0e1bee-3b43-409e-a490-69fb3fb13d3b" (UID: "0e0e1bee-3b43-409e-a490-69fb3fb13d3b"). InnerVolumeSpecName "kube-api-access-97b82". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 22:08:56.686403 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.686364 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97b82\" (UniqueName: \"kubernetes.io/projected/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-kube-api-access-97b82\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:08:56.686403 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.686396 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-maas-api-tls\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:08:56.686403 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.686410 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e0e1bee-3b43-409e-a490-69fb3fb13d3b-openshift-service-ca\") on node \"ip-10-0-139-229.ec2.internal\" DevicePath \"\"" Jul 08 22:08:56.910951 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.910863 2580 generic.go:356] "Generic (PLEG): container finished" podID="0e0e1bee-3b43-409e-a490-69fb3fb13d3b" containerID="8d7637abb8383f5283d5f36f9a4bee4d3c04df64caa1e39759ce6bd8e64f4bdc" exitCode=0 Jul 08 22:08:56.910951 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.910908 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" event={"ID":"0e0e1bee-3b43-409e-a490-69fb3fb13d3b","Type":"ContainerDied","Data":"8d7637abb8383f5283d5f36f9a4bee4d3c04df64caa1e39759ce6bd8e64f4bdc"} Jul 08 22:08:56.910951 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.910917 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" Jul 08 22:08:56.910951 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.910950 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m" event={"ID":"0e0e1bee-3b43-409e-a490-69fb3fb13d3b","Type":"ContainerDied","Data":"3b0843f483d0ee02ca8ec3c17a46c585e1f29d389f63e7fe38d99be6ad9934df"} Jul 08 22:08:56.911699 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.910970 2580 scope.go:117] "RemoveContainer" containerID="8d7637abb8383f5283d5f36f9a4bee4d3c04df64caa1e39759ce6bd8e64f4bdc" Jul 08 22:08:56.920095 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.920079 2580 scope.go:117] "RemoveContainer" containerID="8d7637abb8383f5283d5f36f9a4bee4d3c04df64caa1e39759ce6bd8e64f4bdc" Jul 08 22:08:56.920333 ip-10-0-139-229 kubenswrapper[2580]: E0708 22:08:56.920317 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d7637abb8383f5283d5f36f9a4bee4d3c04df64caa1e39759ce6bd8e64f4bdc\": container with ID starting with 8d7637abb8383f5283d5f36f9a4bee4d3c04df64caa1e39759ce6bd8e64f4bdc not found: ID does not exist" containerID="8d7637abb8383f5283d5f36f9a4bee4d3c04df64caa1e39759ce6bd8e64f4bdc" Jul 08 22:08:56.920394 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.920344 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7637abb8383f5283d5f36f9a4bee4d3c04df64caa1e39759ce6bd8e64f4bdc"} err="failed to get container status \"8d7637abb8383f5283d5f36f9a4bee4d3c04df64caa1e39759ce6bd8e64f4bdc\": rpc error: code = NotFound desc = could not find container \"8d7637abb8383f5283d5f36f9a4bee4d3c04df64caa1e39759ce6bd8e64f4bdc\": container with ID starting with 8d7637abb8383f5283d5f36f9a4bee4d3c04df64caa1e39759ce6bd8e64f4bdc not found: ID does not exist" Jul 08 22:08:56.932759 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.932732 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m"] Jul 08 22:08:56.938294 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:56.938277 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-074c17-7cb797fc7b-fmc2m"] Jul 08 22:08:58.242942 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:08:58.242907 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e0e1bee-3b43-409e-a490-69fb3fb13d3b" path="/var/lib/kubelet/pods/0e0e1bee-3b43-409e-a490-69fb3fb13d3b/volumes" Jul 08 22:09:01.813677 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:01.813645 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:09:01.870974 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:01.870945 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:09:02.099613 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:02.099534 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:09:02.638035 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:02.637997 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:09:02.988499 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:02.988414 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:09:03.088336 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:03.088307 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:09:03.188596 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:03.188565 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:09:03.269629 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:03.269551 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:09:09.183246 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:09.183207 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:09:09.298659 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:09.298624 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:09:09.522403 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:09.522370 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:09:15.101952 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:15.101922 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-2kc5f_57934621-1051-4944-9faf-d715c06d948a/manager/0.log" Jul 08 22:09:15.853271 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:15.853243 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c97f5c75f-jct7c_78ea6aaf-1175-495d-8d84-21178e0b833c/copy-manifests/0.log" Jul 08 22:09:15.859765 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:15.859737 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c97f5c75f-jct7c_78ea6aaf-1175-495d-8d84-21178e0b833c/manager/0.log" Jul 08 22:09:17.649217 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:17.649187 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-ptbrj_a4e7dc17-0bbe-45ef-8ee9-c4f47f3bcd01/manager/0.log" Jul 08 22:09:17.768941 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:17.768914 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-sct7s_df2fd060-9a73-4605-a4a4-f2fa62d61be6/kuadrant-console-plugin/0.log" Jul 08 22:09:18.022396 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:18.022372 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-jvvrx_2dbb2389-5909-4750-99bc-8dfdc353d20e/manager/0.log" Jul 08 22:09:18.876864 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:18.876834 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-589c8d496f-grdph_8e2dd961-2fb7-4da7-b915-bef8ff7825f6/kube-auth-proxy/0.log" Jul 08 22:09:19.948293 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:19.948261 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-jqbzx_d3cf3621-a61b-4181-a7b4-cc7480788086/main/0.log" Jul 08 22:09:20.297037 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:20.297009 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-lhpzb_005898a5-8bda-4179-b470-c8d8ac5a869b/main/0.log" Jul 08 22:09:20.413967 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:20.413936 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-tpgsk_26f60c46-b8a9-48d5-9691-3c2e78df8767/main/0.log" Jul 08 22:09:25.199999 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:25.199969 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:09:25.300005 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:25.299973 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" limit=1024 Jul 08 22:09:28.263166 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:28.263126 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jbjl2_1eac0252-00ff-4edd-a198-158b8a99ba18/global-pull-secret-syncer/0.log" Jul 08 22:09:28.378734 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:28.378703 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bcglp_990847bf-9c74-4725-84b0-fb3c2f05a5c2/konnectivity-agent/0.log" Jul 08 22:09:28.512904 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:28.512876 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-229.ec2.internal_4a1b4940af681a5e34bbf5436c61df86/haproxy/0.log" Jul 08 22:09:33.624319 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:33.624284 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-ptbrj_a4e7dc17-0bbe-45ef-8ee9-c4f47f3bcd01/manager/0.log" Jul 08 22:09:33.665277 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:33.665210 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-sct7s_df2fd060-9a73-4605-a4a4-f2fa62d61be6/kuadrant-console-plugin/0.log" Jul 08 22:09:33.846115 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:33.846086 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-jvvrx_2dbb2389-5909-4750-99bc-8dfdc353d20e/manager/0.log" Jul 08 22:09:35.494523 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:35.494428 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aacc30e8-9af4-43ea-ada9-85fa05d610fc/alertmanager/0.log" Jul 08 22:09:35.518526 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:35.518505 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aacc30e8-9af4-43ea-ada9-85fa05d610fc/config-reloader/0.log" Jul 08 22:09:35.542563 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:35.542534 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aacc30e8-9af4-43ea-ada9-85fa05d610fc/kube-rbac-proxy-web/0.log" Jul 08 22:09:35.574534 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:35.574507 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aacc30e8-9af4-43ea-ada9-85fa05d610fc/kube-rbac-proxy/0.log" Jul 08 22:09:35.597050 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:35.597021 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aacc30e8-9af4-43ea-ada9-85fa05d610fc/kube-rbac-proxy-metric/0.log" Jul 08 22:09:35.621965 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:35.621942 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aacc30e8-9af4-43ea-ada9-85fa05d610fc/prom-label-proxy/0.log" Jul 08 22:09:35.643579 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:35.643555 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aacc30e8-9af4-43ea-ada9-85fa05d610fc/init-config-reloader/0.log" Jul 08 22:09:35.831270 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:35.831242 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7dd59dc87b-ng8wx_19d49c0b-7f35-435d-909b-d4509393af21/metrics-server/0.log" Jul 08 22:09:35.864047 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:35.864022 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-6mp5c_01bee47c-0122-421f-b912-f0d9a8a4efa4/monitoring-plugin/0.log" Jul 08 22:09:36.063951 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:36.063925 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xbxnq_595ca2c2-1d86-4fbf-bc62-96fe59b7fa99/node-exporter/0.log" Jul 08 22:09:36.085948 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:36.085925 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xbxnq_595ca2c2-1d86-4fbf-bc62-96fe59b7fa99/kube-rbac-proxy/0.log" Jul 08 22:09:36.110047 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:36.110026 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xbxnq_595ca2c2-1d86-4fbf-bc62-96fe59b7fa99/init-textfile/0.log" Jul 08 22:09:36.138367 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:36.138344 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-hjnln_5ab77262-7491-43ce-bd89-9bdc245161b7/kube-rbac-proxy-main/0.log" Jul 08 22:09:36.160428 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:36.160409 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-hjnln_5ab77262-7491-43ce-bd89-9bdc245161b7/kube-rbac-proxy-self/0.log" Jul 08 22:09:36.186038 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:36.186015 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-hjnln_5ab77262-7491-43ce-bd89-9bdc245161b7/openshift-state-metrics/0.log" Jul 08 22:09:36.227622 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:36.227602 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9c3397fa-fea0-4750-907c-aee6f78072a5/prometheus/0.log" Jul 08 22:09:36.256710 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:36.256689 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9c3397fa-fea0-4750-907c-aee6f78072a5/config-reloader/0.log" Jul 08 22:09:36.279364 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:36.279343 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9c3397fa-fea0-4750-907c-aee6f78072a5/thanos-sidecar/0.log" Jul 08 22:09:36.300046 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:36.300028 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9c3397fa-fea0-4750-907c-aee6f78072a5/kube-rbac-proxy-web/0.log" Jul 08 22:09:36.321390 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:36.321337 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9c3397fa-fea0-4750-907c-aee6f78072a5/kube-rbac-proxy/0.log" Jul 08 22:09:36.344136 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:36.344121 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9c3397fa-fea0-4750-907c-aee6f78072a5/kube-rbac-proxy-thanos/0.log" Jul 08 22:09:36.365805 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:36.365786 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9c3397fa-fea0-4750-907c-aee6f78072a5/init-config-reloader/0.log" Jul 08 22:09:36.398532 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:36.398508 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-dkc9f_03d072a8-9433-41a7-89c0-0e7d2acb08e0/prometheus-operator/0.log" Jul 08 22:09:36.420464 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:36.420440 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-dkc9f_03d072a8-9433-41a7-89c0-0e7d2acb08e0/kube-rbac-proxy/0.log" Jul 08 22:09:38.644641 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.644606 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh"] Jul 08 22:09:38.645021 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.645005 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e0e1bee-3b43-409e-a490-69fb3fb13d3b" containerName="maas-api" Jul 08 22:09:38.645021 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.645020 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e0e1bee-3b43-409e-a490-69fb3fb13d3b" containerName="maas-api" Jul 08 22:09:38.645097 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.645086 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e0e1bee-3b43-409e-a490-69fb3fb13d3b" containerName="maas-api" Jul 08 22:09:38.648071 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.648055 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" Jul 08 22:09:38.650380 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.650353 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pfvkg\"/\"openshift-service-ca.crt\"" Jul 08 22:09:38.650380 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.650368 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pfvkg\"/\"default-dockercfg-qjbz2\"" Jul 08 22:09:38.650541 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.650439 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pfvkg\"/\"kube-root-ca.crt\"" Jul 08 22:09:38.656873 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.656849 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh"] Jul 08 22:09:38.755501 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.755473 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93ef7cc8-8d11-4410-8399-12ea7a631854-sys\") pod \"perf-node-gather-daemonset-vmgdh\" (UID: \"93ef7cc8-8d11-4410-8399-12ea7a631854\") " pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" Jul 08 22:09:38.755501 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.755503 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/93ef7cc8-8d11-4410-8399-12ea7a631854-podres\") pod \"perf-node-gather-daemonset-vmgdh\" (UID: \"93ef7cc8-8d11-4410-8399-12ea7a631854\") " pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" Jul 08 22:09:38.755740 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.755538 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/93ef7cc8-8d11-4410-8399-12ea7a631854-lib-modules\") pod \"perf-node-gather-daemonset-vmgdh\" (UID: \"93ef7cc8-8d11-4410-8399-12ea7a631854\") " pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" Jul 08 22:09:38.755740 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.755614 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/93ef7cc8-8d11-4410-8399-12ea7a631854-proc\") pod \"perf-node-gather-daemonset-vmgdh\" (UID: \"93ef7cc8-8d11-4410-8399-12ea7a631854\") " pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" Jul 08 22:09:38.755740 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.755686 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwth2\" (UniqueName: \"kubernetes.io/projected/93ef7cc8-8d11-4410-8399-12ea7a631854-kube-api-access-vwth2\") pod \"perf-node-gather-daemonset-vmgdh\" (UID: \"93ef7cc8-8d11-4410-8399-12ea7a631854\") " pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" Jul 08 22:09:38.856733 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.856702 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/93ef7cc8-8d11-4410-8399-12ea7a631854-proc\") pod \"perf-node-gather-daemonset-vmgdh\" (UID: \"93ef7cc8-8d11-4410-8399-12ea7a631854\") " pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" Jul 08 22:09:38.856916 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.856749 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwth2\" (UniqueName: \"kubernetes.io/projected/93ef7cc8-8d11-4410-8399-12ea7a631854-kube-api-access-vwth2\") pod \"perf-node-gather-daemonset-vmgdh\" (UID: \"93ef7cc8-8d11-4410-8399-12ea7a631854\") " pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" Jul 08 22:09:38.856916 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.856779 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93ef7cc8-8d11-4410-8399-12ea7a631854-sys\") pod \"perf-node-gather-daemonset-vmgdh\" (UID: \"93ef7cc8-8d11-4410-8399-12ea7a631854\") " pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" Jul 08 22:09:38.856916 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.856796 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/93ef7cc8-8d11-4410-8399-12ea7a631854-podres\") pod \"perf-node-gather-daemonset-vmgdh\" (UID: \"93ef7cc8-8d11-4410-8399-12ea7a631854\") " pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" Jul 08 22:09:38.856916 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.856840 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/93ef7cc8-8d11-4410-8399-12ea7a631854-proc\") pod \"perf-node-gather-daemonset-vmgdh\" (UID: \"93ef7cc8-8d11-4410-8399-12ea7a631854\") " pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" Jul 08 22:09:38.856916 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.856874 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/93ef7cc8-8d11-4410-8399-12ea7a631854-lib-modules\") pod \"perf-node-gather-daemonset-vmgdh\" (UID: \"93ef7cc8-8d11-4410-8399-12ea7a631854\") " pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" Jul 08 22:09:38.856916 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.856895 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93ef7cc8-8d11-4410-8399-12ea7a631854-sys\") pod \"perf-node-gather-daemonset-vmgdh\" (UID: \"93ef7cc8-8d11-4410-8399-12ea7a631854\") " pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" Jul 08 22:09:38.857195 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.856938 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/93ef7cc8-8d11-4410-8399-12ea7a631854-podres\") pod \"perf-node-gather-daemonset-vmgdh\" (UID: \"93ef7cc8-8d11-4410-8399-12ea7a631854\") " pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" Jul 08 22:09:38.857195 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.857000 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/93ef7cc8-8d11-4410-8399-12ea7a631854-lib-modules\") pod \"perf-node-gather-daemonset-vmgdh\" (UID: \"93ef7cc8-8d11-4410-8399-12ea7a631854\") " pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" Jul 08 22:09:38.865067 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.865045 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwth2\" (UniqueName: \"kubernetes.io/projected/93ef7cc8-8d11-4410-8399-12ea7a631854-kube-api-access-vwth2\") pod \"perf-node-gather-daemonset-vmgdh\" (UID: \"93ef7cc8-8d11-4410-8399-12ea7a631854\") " pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" Jul 08 22:09:38.958805 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:38.958730 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" Jul 08 22:09:39.091347 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:39.091308 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh"] Jul 08 22:09:39.094202 ip-10-0-139-229 kubenswrapper[2580]: W0708 22:09:39.094176 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod93ef7cc8_8d11_4410_8399_12ea7a631854.slice/crio-5796ccbef4d380799ad93a4eef62c06cfb1772f7b8b9dc18ad999c9e61df5165 WatchSource:0}: Error finding container 5796ccbef4d380799ad93a4eef62c06cfb1772f7b8b9dc18ad999c9e61df5165: Status 404 returned error can't find the container with id 5796ccbef4d380799ad93a4eef62c06cfb1772f7b8b9dc18ad999c9e61df5165 Jul 08 22:09:39.100366 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:39.100322 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" event={"ID":"93ef7cc8-8d11-4410-8399-12ea7a631854","Type":"ContainerStarted","Data":"5796ccbef4d380799ad93a4eef62c06cfb1772f7b8b9dc18ad999c9e61df5165"} Jul 08 22:09:40.107179 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:40.107146 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" event={"ID":"93ef7cc8-8d11-4410-8399-12ea7a631854","Type":"ContainerStarted","Data":"db7046fb5fc302b434970f488aba2bbeb2296696ab50a43b6de116728277e7ec"} Jul 08 22:09:40.107557 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:40.107274 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" Jul 08 22:09:40.123568 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:40.123516 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" podStartSLOduration=2.123502098 podStartE2EDuration="2.123502098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 22:09:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 22:09:40.121802036 +0000 UTC m=+2202.419686985" watchObservedRunningTime="2026-07-08 22:09:40.123502098 +0000 UTC m=+2202.421387032" Jul 08 22:09:40.811034 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:40.811003 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cngfd_bc6e0d63-2587-450e-9001-9e1a13874ad6/dns/0.log" Jul 08 22:09:40.830936 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:40.830914 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cngfd_bc6e0d63-2587-450e-9001-9e1a13874ad6/kube-rbac-proxy/0.log" Jul 08 22:09:40.966080 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:40.966053 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gdk2v_b068ba38-cb80-4fa3-b474-294c4d2876a9/dns-node-resolver/0.log" Jul 08 22:09:41.614748 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:41.614720 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vpcft_e4803b19-4882-41f4-98a7-129d0493c630/node-ca/0.log" Jul 08 22:09:42.767854 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:42.767825 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-589c8d496f-grdph_8e2dd961-2fb7-4da7-b915-bef8ff7825f6/kube-auth-proxy/0.log" Jul 08 22:09:43.854919 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:43.854888 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wdmtl_75941359-0abf-4810-87c8-76ea8cc35f65/serve-healthcheck-canary/0.log" Jul 08 22:09:44.355928 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:44.355900 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dpk92_c4075449-512f-49ac-9cdf-65c97d14f19a/kube-rbac-proxy/0.log" Jul 08 22:09:44.375515 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:44.375491 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dpk92_c4075449-512f-49ac-9cdf-65c97d14f19a/exporter/0.log" Jul 08 22:09:44.395709 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:44.395690 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dpk92_c4075449-512f-49ac-9cdf-65c97d14f19a/extractor/0.log" Jul 08 22:09:46.121316 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:46.121283 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pfvkg/perf-node-gather-daemonset-vmgdh" Jul 08 22:09:46.662294 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:46.662265 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-2kc5f_57934621-1051-4944-9faf-d715c06d948a/manager/0.log" Jul 08 22:09:47.259131 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:47.259108 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c97f5c75f-jct7c_78ea6aaf-1175-495d-8d84-21178e0b833c/manager/0.log" Jul 08 22:09:47.283958 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:47.283927 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c97f5c75f-jct7c_78ea6aaf-1175-495d-8d84-21178e0b833c/copy-manifests/0.log" Jul 08 22:09:48.879086 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:48.879063 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-857546b9c4-v886x_aaadcb70-0dcf-48a6-aafa-27bec22ba179/manager/0.log" Jul 08 22:09:49.548383 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:49.548355 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-589c8d496f-grdph_8e2dd961-2fb7-4da7-b915-bef8ff7825f6/kube-auth-proxy/0.log" Jul 08 22:09:51.205600 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:51.205572 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-589c8d496f-grdph_8e2dd961-2fb7-4da7-b915-bef8ff7825f6/kube-auth-proxy/0.log" Jul 08 22:09:55.360895 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:55.360866 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9p7zn_09c797d3-78e5-472e-b000-c66f6075ba77/kube-multus-additional-cni-plugins/0.log" Jul 08 22:09:55.382205 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:55.382171 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9p7zn_09c797d3-78e5-472e-b000-c66f6075ba77/egress-router-binary-copy/0.log" Jul 08 22:09:55.401629 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:55.401601 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9p7zn_09c797d3-78e5-472e-b000-c66f6075ba77/cni-plugins/0.log" Jul 08 22:09:55.420775 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:55.420751 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9p7zn_09c797d3-78e5-472e-b000-c66f6075ba77/bond-cni-plugin/0.log" Jul 08 22:09:55.439634 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:55.439615 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9p7zn_09c797d3-78e5-472e-b000-c66f6075ba77/routeoverride-cni/0.log" Jul 08 22:09:55.458753 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:55.458732 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9p7zn_09c797d3-78e5-472e-b000-c66f6075ba77/whereabouts-cni-bincopy/0.log" Jul 08 22:09:55.477567 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:55.477539 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9p7zn_09c797d3-78e5-472e-b000-c66f6075ba77/whereabouts-cni/0.log" Jul 08 22:09:55.849738 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:55.849709 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vlzqf_dcc5996d-a926-4802-853f-c4991d8ecbef/kube-multus/0.log" Jul 08 22:09:55.912677 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:55.912646 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8bgg4_57804798-9f60-42ad-82f9-8ff4065af152/network-metrics-daemon/0.log" Jul 08 22:09:55.932403 ip-10-0-139-229 kubenswrapper[2580]: I0708 22:09:55.932383 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8bgg4_57804798-9f60-42ad-82f9-8ff4065af152/kube-rbac-proxy/0.log"