Jul 03 10:03:20.531265 ip-10-0-134-191 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 03 10:03:20.531273 ip-10-0-134-191 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 03 10:03:20.531280 ip-10-0-134-191 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 03 10:03:20.531495 ip-10-0-134-191 systemd[1]: Failed to start Kubernetes Kubelet. Jul 03 10:03:31.904556 ip-10-0-134-191 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 03 10:03:31.904572 ip-10-0-134-191 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot e36ff992abaa4169ac23c0dad4ddc4d0 -- Jul 03 10:05:44.970000 ip-10-0-134-191 systemd[1]: Starting Kubernetes Kubelet... Jul 03 10:05:45.415257 ip-10-0-134-191 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 03 10:05:45.415257 ip-10-0-134-191 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 03 10:05:45.415257 ip-10-0-134-191 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 03 10:05:45.415257 ip-10-0-134-191 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 03 10:05:45.415257 ip-10-0-134-191 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 03 10:05:45.415257 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.373652 2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 03 10:05:45.415257 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378108 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 10:05:45.415257 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378121 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 10:05:45.415257 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378124 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 10:05:45.415257 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378129 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 10:05:45.415257 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378132 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 10:05:45.415257 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378135 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 10:05:45.415257 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378139 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 10:05:45.415257 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378141 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 10:05:45.416657 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378144 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 10:05:45.416657 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378147 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 10:05:45.416657 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378150 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 10:05:45.416657 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378152 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 10:05:45.416657 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378155 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 10:05:45.416657 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378157 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 10:05:45.416657 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378161 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 10:05:45.416657 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378164 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 10:05:45.416657 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378167 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 10:05:45.416657 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378169 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 10:05:45.416657 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378172 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 10:05:45.416657 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378174 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 10:05:45.416657 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378177 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 10:05:45.416657 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378185 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 10:05:45.416657 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378188 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 10:05:45.416657 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378191 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 10:05:45.416657 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378193 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 10:05:45.416657 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378196 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 10:05:45.416657 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378200 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 10:05:45.416657 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378203 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 10:05:45.417208 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378205 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 10:05:45.417208 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378208 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 10:05:45.417208 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378210 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 10:05:45.417208 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378215 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 10:05:45.417208 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378219 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 10:05:45.417208 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378222 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 10:05:45.417208 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378225 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 10:05:45.417208 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378228 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 10:05:45.417208 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378231 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 10:05:45.417208 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378234 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 10:05:45.417208 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378237 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 10:05:45.417208 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378239 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 10:05:45.417208 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378242 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 10:05:45.417208 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378245 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 10:05:45.417208 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378249 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 10:05:45.417208 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378251 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 10:05:45.417208 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378254 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 10:05:45.417208 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378256 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 10:05:45.417208 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378260 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 10:05:45.417871 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378262 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 10:05:45.417871 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378265 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 10:05:45.417871 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378267 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 10:05:45.417871 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378270 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 10:05:45.417871 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378273 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 10:05:45.417871 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378276 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 10:05:45.417871 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378279 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 10:05:45.417871 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378281 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 10:05:45.417871 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378284 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 10:05:45.417871 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378286 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 10:05:45.417871 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378289 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 10:05:45.417871 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378292 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 10:05:45.417871 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378295 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 10:05:45.417871 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378298 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 10:05:45.417871 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378300 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 10:05:45.417871 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378303 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 10:05:45.417871 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378306 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 10:05:45.417871 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378308 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 10:05:45.417871 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378310 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 10:05:45.417871 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378313 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 10:05:45.418457 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378316 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 10:05:45.418457 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378318 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 10:05:45.418457 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378323 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 10:05:45.418457 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378326 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 10:05:45.418457 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378329 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 10:05:45.418457 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378332 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 10:05:45.418457 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378335 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 10:05:45.418457 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378337 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 10:05:45.418457 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378339 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 10:05:45.418457 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378342 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 10:05:45.418457 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378344 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 10:05:45.418457 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378347 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 10:05:45.418457 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378349 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 10:05:45.418457 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378352 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 03 10:05:45.418457 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378356 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 10:05:45.418457 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378359 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 10:05:45.418457 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378363 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 10:05:45.418457 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378366 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 10:05:45.418457 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378368 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 10:05:45.419211 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378372 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 10:05:45.419211 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378375 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 10:05:45.419211 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378378 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 10:05:45.419211 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378381 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 10:05:45.419211 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378384 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 10:05:45.419211 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378387 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 10:05:45.419211 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378390 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 10:05:45.419211 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378393 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 10:05:45.419211 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378396 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 10:05:45.419211 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378399 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 10:05:45.419211 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378401 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 10:05:45.419211 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378404 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 10:05:45.419211 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.378406 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 10:05:45.419211 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379168 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 10:05:45.419211 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379176 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 10:05:45.419211 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379178 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 10:05:45.419211 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379181 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 10:05:45.419211 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379184 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 10:05:45.419211 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379187 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 10:05:45.420213 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379190 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 10:05:45.420213 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379193 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 10:05:45.420213 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379196 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 10:05:45.420213 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379198 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 10:05:45.420213 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379201 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 10:05:45.420213 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379204 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 10:05:45.420213 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379207 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 10:05:45.420213 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379210 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 10:05:45.420213 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379213 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 10:05:45.420213 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379216 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 10:05:45.420213 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379219 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 10:05:45.420213 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379222 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 10:05:45.420213 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379224 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 10:05:45.420213 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379227 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 10:05:45.420213 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379229 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 10:05:45.420213 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379232 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 10:05:45.420213 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379234 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 10:05:45.420213 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379237 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 10:05:45.420213 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379239 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 10:05:45.420213 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379242 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 10:05:45.420814 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379245 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 10:05:45.420814 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379247 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 10:05:45.420814 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379250 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 10:05:45.420814 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379252 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 10:05:45.420814 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379255 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 10:05:45.420814 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379258 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 10:05:45.420814 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379262 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 10:05:45.420814 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379266 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 10:05:45.420814 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379269 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 10:05:45.420814 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379271 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 10:05:45.420814 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379274 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 10:05:45.420814 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379276 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 03 10:05:45.420814 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379279 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 10:05:45.420814 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379282 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 10:05:45.420814 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379285 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 10:05:45.420814 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379287 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 10:05:45.420814 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379289 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 10:05:45.420814 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379292 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 10:05:45.420814 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379295 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 10:05:45.420814 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379297 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 10:05:45.421758 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379300 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 10:05:45.421758 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379302 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 10:05:45.421758 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379305 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 10:05:45.421758 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379307 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 10:05:45.421758 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379310 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 10:05:45.421758 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379312 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 10:05:45.421758 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379315 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 10:05:45.421758 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379318 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 10:05:45.421758 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379321 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 10:05:45.421758 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379323 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 10:05:45.421758 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379326 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 10:05:45.421758 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379328 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 10:05:45.421758 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379331 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 10:05:45.421758 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379333 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 10:05:45.421758 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379336 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 10:05:45.421758 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379338 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 10:05:45.421758 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379341 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 10:05:45.421758 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379343 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 10:05:45.421758 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379346 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 10:05:45.421758 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379348 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 10:05:45.532761 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379351 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 10:05:45.532761 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379353 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 10:05:45.532761 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379356 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 10:05:45.532761 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379359 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 10:05:45.532761 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379361 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 10:05:45.532761 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379364 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 10:05:45.532761 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379367 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 10:05:45.532761 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379369 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 10:05:45.532761 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379372 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 10:05:45.532761 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379375 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 10:05:45.532761 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379377 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 10:05:45.532761 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379383 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 10:05:45.532761 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379387 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 10:05:45.532761 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379390 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 10:05:45.532761 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379394 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 10:05:45.532761 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379398 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 10:05:45.532761 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379402 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 10:05:45.532761 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379406 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 10:05:45.532761 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379410 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 10:05:45.449151 ip-10-0-134-191 systemd[1]: Started Kubernetes Kubelet. Jul 03 10:05:45.533899 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379413 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 10:05:45.533899 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379416 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 10:05:45.533899 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379419 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 10:05:45.533899 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379422 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 10:05:45.533899 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379424 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 10:05:45.533899 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379427 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 10:05:45.533899 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379429 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 10:05:45.533899 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379432 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 10:05:45.533899 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379434 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 10:05:45.533899 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379437 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 10:05:45.533899 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379439 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 10:05:45.533899 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379442 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 10:05:45.533899 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379444 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 10:05:45.533899 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379447 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 10:05:45.533899 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379838 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 10:05:45.533899 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379843 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 10:05:45.533899 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379846 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 10:05:45.533899 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379850 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 10:05:45.533899 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379854 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 10:05:45.535150 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379857 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 10:05:45.535150 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379860 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 10:05:45.535150 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379863 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 10:05:45.535150 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379866 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 10:05:45.535150 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379869 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 10:05:45.535150 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379873 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 10:05:45.535150 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379876 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 10:05:45.535150 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379878 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 10:05:45.535150 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379881 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 10:05:45.535150 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379883 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 10:05:45.535150 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379886 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 10:05:45.535150 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379890 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 10:05:45.535150 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379892 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 10:05:45.535150 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379896 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 10:05:45.535150 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379899 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 10:05:45.535150 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379902 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 10:05:45.535150 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379905 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 10:05:45.535150 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379908 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 10:05:45.535150 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379911 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 10:05:45.538061 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379913 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 10:05:45.538061 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379916 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 10:05:45.538061 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379918 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 10:05:45.538061 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379921 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 10:05:45.538061 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379924 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 10:05:45.538061 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379926 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 10:05:45.538061 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379929 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 10:05:45.538061 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379931 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 10:05:45.538061 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379934 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 10:05:45.538061 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379936 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 10:05:45.538061 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379939 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 10:05:45.538061 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379941 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 10:05:45.538061 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379943 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 10:05:45.538061 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379947 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 10:05:45.538061 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379949 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 10:05:45.538061 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379952 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 10:05:45.538061 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379954 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 10:05:45.538061 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379957 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 10:05:45.538061 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379961 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 10:05:45.538061 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379963 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 10:05:45.539384 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379966 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 10:05:45.539384 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379969 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 10:05:45.539384 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379974 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 10:05:45.539384 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379977 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 10:05:45.539384 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379981 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 10:05:45.539384 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379984 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 10:05:45.539384 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379986 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 10:05:45.539384 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379988 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 10:05:45.539384 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379991 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 10:05:45.539384 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379994 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 10:05:45.539384 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379996 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 10:05:45.539384 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.379999 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 10:05:45.539384 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380001 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 10:05:45.539384 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380003 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 10:05:45.539384 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380006 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 10:05:45.539384 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380009 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 10:05:45.539384 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380011 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 10:05:45.539384 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380014 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 10:05:45.539384 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380016 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 10:05:45.539384 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380020 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 10:05:45.541512 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380022 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 10:05:45.541512 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380026 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 10:05:45.541512 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380029 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 10:05:45.541512 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380032 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 10:05:45.541512 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380035 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 10:05:45.541512 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380037 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 10:05:45.541512 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380040 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 10:05:45.541512 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380044 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 10:05:45.541512 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380046 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 10:05:45.541512 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380049 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 10:05:45.541512 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380052 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 10:05:45.541512 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380055 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 10:05:45.541512 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380057 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 10:05:45.541512 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380060 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 10:05:45.541512 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380062 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 10:05:45.541512 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380065 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 10:05:45.541512 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380068 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 10:05:45.541512 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380071 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 10:05:45.541512 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380073 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 10:05:45.541512 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380076 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 10:05:45.542477 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380078 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 10:05:45.542477 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380081 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 10:05:45.542477 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380083 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 10:05:45.542477 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380086 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 10:05:45.542477 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380088 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 10:05:45.542477 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380091 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 10:05:45.542477 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380094 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 10:05:45.542477 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380096 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 10:05:45.542477 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380099 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 03 10:05:45.542477 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380101 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 10:05:45.542477 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380104 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 10:05:45.542477 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380107 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 10:05:45.542477 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380109 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 10:05:45.542477 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380111 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 10:05:45.542477 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380114 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 10:05:45.542477 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380182 2573 flags.go:64] FLAG: --address="0.0.0.0" Jul 03 10:05:45.542477 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380189 2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 03 10:05:45.542477 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380195 2573 flags.go:64] FLAG: --anonymous-auth="true" Jul 03 10:05:45.542477 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380200 2573 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 03 10:05:45.542477 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380209 2573 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380212 2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380216 2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380222 2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380225 2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380228 2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380232 2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380235 2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380238 2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380243 2573 flags.go:64] FLAG: --cgroup-root="" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380246 2573 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380249 2573 flags.go:64] FLAG: --client-ca-file="" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380252 2573 flags.go:64] FLAG: --cloud-provider="external" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380255 2573 flags.go:64] FLAG: --cluster-dns="[]" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380259 2573 flags.go:64] FLAG: --cluster-domain="" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380262 2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380265 2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380268 2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380272 2573 flags.go:64] FLAG: --container-log-max-files="5" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380275 2573 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380278 2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380282 2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380285 2573 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380288 2573 flags.go:64] FLAG: --contention-profiling="false" Jul 03 10:05:45.544599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380291 2573 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380294 2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380297 2573 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380299 2573 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380303 2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380306 2573 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380309 2573 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380312 2573 flags.go:64] FLAG: --enable-load-reader="false" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380315 2573 flags.go:64] FLAG: --enable-server="true" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380319 2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380324 2573 flags.go:64] FLAG: --event-burst="100" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380328 2573 flags.go:64] FLAG: --event-qps="50" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380331 2573 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380334 2573 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380337 2573 flags.go:64] FLAG: --eviction-hard="" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380341 2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380344 2573 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380348 2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380351 2573 flags.go:64] FLAG: --eviction-soft="" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380354 2573 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380357 2573 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380360 2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380362 2573 flags.go:64] FLAG: --experimental-mounter-path="" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380365 2573 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380368 2573 flags.go:64] FLAG: --fail-swap-on="true" Jul 03 10:05:45.545939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380371 2573 flags.go:64] FLAG: --feature-gates="" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380375 2573 flags.go:64] FLAG: --file-check-frequency="20s" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380378 2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380381 2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380384 2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380387 2573 flags.go:64] FLAG: --healthz-port="10248" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380390 2573 flags.go:64] FLAG: --help="false" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380393 2573 flags.go:64] FLAG: --hostname-override="ip-10-0-134-191.ec2.internal" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380396 2573 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380399 2573 flags.go:64] FLAG: --http-check-frequency="20s" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380402 2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380405 2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380408 2573 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380411 2573 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380414 2573 flags.go:64] FLAG: --image-service-endpoint="" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380417 2573 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380419 2573 flags.go:64] FLAG: --kube-api-burst="100" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380423 2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380427 2573 flags.go:64] FLAG: --kube-api-qps="50" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380430 2573 flags.go:64] FLAG: --kube-reserved="" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380433 2573 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380435 2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380438 2573 flags.go:64] FLAG: --kubelet-cgroups="" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380441 2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 03 10:05:45.547261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380445 2573 flags.go:64] FLAG: --lock-file="" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380448 2573 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380451 2573 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380454 2573 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380459 2573 flags.go:64] FLAG: --log-json-split-stream="false" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380462 2573 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380465 2573 flags.go:64] FLAG: --log-text-split-stream="false" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380468 2573 flags.go:64] FLAG: --logging-format="text" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380471 2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380474 2573 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380477 2573 flags.go:64] FLAG: --manifest-url="" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380480 2573 flags.go:64] FLAG: --manifest-url-header="" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380484 2573 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380487 2573 flags.go:64] FLAG: --max-open-files="1000000" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380491 2573 flags.go:64] FLAG: --max-pods="110" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380495 2573 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380497 2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380501 2573 flags.go:64] FLAG: --memory-manager-policy="None" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380504 2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380507 2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380509 2573 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380512 2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380520 2573 flags.go:64] FLAG: --node-status-max-images="50" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380523 2573 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380526 2573 flags.go:64] FLAG: --oom-score-adj="-999" Jul 03 10:05:45.548638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380528 2573 flags.go:64] FLAG: --pod-cidr="" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380537 2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380543 2573 flags.go:64] FLAG: --pod-manifest-path="" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380546 2573 flags.go:64] FLAG: --pod-max-pids="-1" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380549 2573 flags.go:64] FLAG: --pods-per-core="0" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380552 2573 flags.go:64] FLAG: --port="10250" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380556 2573 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380560 2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-031478e643c42f7ad" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380563 2573 flags.go:64] FLAG: --qos-reserved="" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380566 2573 flags.go:64] FLAG: --read-only-port="10255" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380569 2573 flags.go:64] FLAG: --register-node="true" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380572 2573 flags.go:64] FLAG: --register-with-taints="" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380576 2573 flags.go:64] FLAG: --registry-burst="10" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380580 2573 flags.go:64] FLAG: --registry-qps="5" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380582 2573 flags.go:64] FLAG: --reserved-cpus="" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380585 2573 flags.go:64] FLAG: --reserved-memory="" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380589 2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380592 2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380595 2573 flags.go:64] FLAG: --rotate-certificates="false" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380597 2573 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380600 2573 flags.go:64] FLAG: --runonce="false" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380603 2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380606 2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380609 2573 flags.go:64] FLAG: --seccomp-default="false" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380612 2573 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 03 10:05:45.549995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380614 2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380618 2573 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380621 2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380624 2573 flags.go:64] FLAG: --storage-driver-password="root" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380627 2573 flags.go:64] FLAG: --storage-driver-secure="false" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380630 2573 flags.go:64] FLAG: --storage-driver-table="stats" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380633 2573 flags.go:64] FLAG: --storage-driver-user="root" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380635 2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380638 2573 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380644 2573 flags.go:64] FLAG: --system-cgroups="" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380646 2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380652 2573 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380654 2573 flags.go:64] FLAG: --tls-cert-file="" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380657 2573 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380663 2573 flags.go:64] FLAG: --tls-min-version="" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380666 2573 flags.go:64] FLAG: --tls-private-key-file="" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380669 2573 flags.go:64] FLAG: --topology-manager-policy="none" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380672 2573 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380674 2573 flags.go:64] FLAG: --topology-manager-scope="container" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380678 2573 flags.go:64] FLAG: --v="2" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380681 2573 flags.go:64] FLAG: --version="false" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380685 2573 flags.go:64] FLAG: --vmodule="" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380689 2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.380692 2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 03 10:05:45.551344 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380815 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380821 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380825 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380828 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380831 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380833 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380836 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380839 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380842 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380844 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380847 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380849 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380852 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380854 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380857 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380860 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380862 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380867 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380870 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380873 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380876 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 10:05:45.552602 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380878 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 10:05:45.553551 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380882 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 10:05:45.553551 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380885 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 10:05:45.553551 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380887 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 10:05:45.553551 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380890 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 10:05:45.553551 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380892 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 10:05:45.553551 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380895 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 10:05:45.553551 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380897 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 10:05:45.553551 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380900 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 10:05:45.553551 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380902 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 10:05:45.553551 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380905 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 10:05:45.553551 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380907 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 10:05:45.553551 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380910 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 10:05:45.553551 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380912 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 10:05:45.553551 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380915 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 10:05:45.553551 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380917 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 10:05:45.553551 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380920 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 10:05:45.553551 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380922 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 10:05:45.553551 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380925 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 10:05:45.553551 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380927 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 10:05:45.554473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380930 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 10:05:45.554473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380932 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 10:05:45.554473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380935 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 10:05:45.554473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380937 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 10:05:45.554473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380939 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 10:05:45.554473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380942 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 10:05:45.554473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380945 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 10:05:45.554473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380947 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 10:05:45.554473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380951 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 10:05:45.554473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380953 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 10:05:45.554473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380956 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 10:05:45.554473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380959 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 10:05:45.554473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380962 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 10:05:45.554473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380966 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 10:05:45.554473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380969 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 10:05:45.554473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380971 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 10:05:45.554473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380974 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 10:05:45.554473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380977 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 10:05:45.554473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380979 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 10:05:45.554473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380981 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 10:05:45.555397 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380984 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 10:05:45.555397 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380986 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 10:05:45.555397 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380989 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 10:05:45.555397 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380991 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 10:05:45.555397 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380994 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 10:05:45.555397 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380996 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 10:05:45.555397 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.380999 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 10:05:45.555397 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381002 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 10:05:45.555397 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381004 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 10:05:45.555397 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381007 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 10:05:45.555397 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381009 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 10:05:45.555397 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381013 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 10:05:45.555397 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381017 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 10:05:45.555397 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381021 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 10:05:45.555397 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381023 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 10:05:45.555397 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381026 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 10:05:45.555397 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381028 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 10:05:45.555397 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381031 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 10:05:45.555397 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381034 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 10:05:45.556456 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381036 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 10:05:45.556456 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381041 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 10:05:45.556456 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381043 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 10:05:45.556456 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381046 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 10:05:45.556456 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381049 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 10:05:45.556456 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381053 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 10:05:45.556456 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381058 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 10:05:45.556456 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381061 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 10:05:45.556456 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381064 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 10:05:45.556456 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381067 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 10:05:45.556456 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381070 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 10:05:45.556456 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381073 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 10:05:45.556456 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381076 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 10:05:45.556456 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381078 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 10:05:45.556456 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381081 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 03 10:05:45.556456 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381084 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 10:05:45.556456 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381086 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 10:05:45.556456 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381089 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 10:05:45.556456 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.381091 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 10:05:45.557349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.381700 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 03 10:05:45.557349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.388243 2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 03 10:05:45.557349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.388259 2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 03 10:05:45.557349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.388868 2573 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 03 10:05:45.557349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.388879 2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 03 10:05:45.557349 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.388934 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 10:05:45.557349 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.388941 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 10:05:45.557349 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.388945 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 10:05:45.557349 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.388949 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 10:05:45.557349 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.388952 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 10:05:45.557349 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.388955 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 10:05:45.557349 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.388958 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 10:05:45.557349 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.388961 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 10:05:45.557349 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.388964 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 10:05:45.559439 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.388967 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 10:05:45.559439 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.388971 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 10:05:45.559439 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.388975 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 10:05:45.559439 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.388982 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 10:05:45.559439 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.388985 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 10:05:45.559439 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.388988 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 10:05:45.559439 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.388991 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 10:05:45.559439 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.388994 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 10:05:45.559439 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.388997 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 10:05:45.559439 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389000 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 10:05:45.559439 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389003 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 10:05:45.559439 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389005 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 10:05:45.559439 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389008 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 10:05:45.559439 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389012 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 10:05:45.559439 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389014 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 10:05:45.559439 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389017 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 10:05:45.559439 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389019 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 10:05:45.559439 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389022 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 10:05:45.559439 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389025 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 10:05:45.559439 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389027 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 10:05:45.560354 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389030 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 10:05:45.560354 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389033 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 10:05:45.560354 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389035 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 10:05:45.560354 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389038 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 10:05:45.560354 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389043 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 10:05:45.560354 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389045 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 10:05:45.560354 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389048 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 10:05:45.560354 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389051 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 10:05:45.560354 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389053 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 10:05:45.560354 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389056 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 10:05:45.560354 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389059 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 10:05:45.560354 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389062 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 10:05:45.560354 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389064 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 10:05:45.560354 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389067 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 10:05:45.560354 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389070 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 10:05:45.560354 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389072 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 10:05:45.560354 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389076 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 10:05:45.560354 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389079 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 10:05:45.560354 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389081 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 10:05:45.560354 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389084 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 10:05:45.562900 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389086 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 10:05:45.562900 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389089 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 10:05:45.562900 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389092 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 10:05:45.562900 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389094 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 10:05:45.562900 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389097 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 10:05:45.562900 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389099 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 10:05:45.562900 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389101 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 10:05:45.562900 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389104 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 10:05:45.562900 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389107 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 10:05:45.562900 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389109 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 10:05:45.562900 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389112 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 10:05:45.562900 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389115 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 10:05:45.562900 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389117 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 10:05:45.562900 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389120 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 10:05:45.562900 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389122 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 10:05:45.562900 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389127 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 10:05:45.562900 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389130 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 10:05:45.562900 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389133 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 10:05:45.562900 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389136 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 10:05:45.563825 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389139 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 10:05:45.563825 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389141 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 10:05:45.563825 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389144 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 10:05:45.563825 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389146 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 10:05:45.563825 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389149 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 10:05:45.563825 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389152 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 10:05:45.563825 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389155 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 10:05:45.563825 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389157 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 10:05:45.563825 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389160 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 10:05:45.563825 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389162 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 10:05:45.563825 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389167 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 10:05:45.563825 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389170 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 10:05:45.563825 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389173 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 10:05:45.563825 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389176 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 10:05:45.563825 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389179 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 10:05:45.563825 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389182 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 10:05:45.563825 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389185 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 10:05:45.563825 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389187 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 10:05:45.563825 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389191 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 10:05:45.652755 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389193 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 10:05:45.652755 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389196 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 10:05:45.652755 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389198 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 10:05:45.652755 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389201 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 10:05:45.652755 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389203 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 10:05:45.652755 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389206 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 03 10:05:45.652755 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389208 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 10:05:45.652755 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389211 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 10:05:45.652755 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389213 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 10:05:45.652755 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389217 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 10:05:45.652755 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389221 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 10:05:45.652755 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389224 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 10:05:45.652755 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.389229 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 03 10:05:45.652755 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389373 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 03 10:05:45.652755 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389378 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 03 10:05:45.653592 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389381 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 03 10:05:45.653592 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389384 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 03 10:05:45.653592 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389387 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 03 10:05:45.653592 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389390 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 03 10:05:45.653592 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389392 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 03 10:05:45.653592 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389395 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 03 10:05:45.653592 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389398 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 03 10:05:45.653592 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389400 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 03 10:05:45.653592 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389403 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 03 10:05:45.653592 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389406 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 03 10:05:45.653592 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389409 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 03 10:05:45.653592 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389411 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 03 10:05:45.653592 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389414 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 03 10:05:45.653592 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389416 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 03 10:05:45.653592 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389419 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 03 10:05:45.653592 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389421 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 03 10:05:45.653592 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389424 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 03 10:05:45.653592 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389426 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 03 10:05:45.653592 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389429 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 03 10:05:45.653592 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389431 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 03 10:05:45.654539 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389434 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 03 10:05:45.654539 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389436 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 03 10:05:45.654539 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389439 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 03 10:05:45.654539 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389441 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 03 10:05:45.654539 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389443 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 03 10:05:45.654539 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389446 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 03 10:05:45.654539 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389450 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 03 10:05:45.654539 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389453 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 03 10:05:45.654539 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389457 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 03 10:05:45.654539 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389459 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 03 10:05:45.654539 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389462 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 03 10:05:45.654539 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389465 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 03 10:05:45.654539 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389467 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 03 10:05:45.654539 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389471 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 03 10:05:45.654539 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389474 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 03 10:05:45.654539 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389477 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 03 10:05:45.654539 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389480 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 03 10:05:45.654539 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389483 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 03 10:05:45.654539 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389486 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 03 10:05:45.655444 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389489 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 03 10:05:45.655444 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389491 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 03 10:05:45.655444 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389494 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 03 10:05:45.655444 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389498 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 03 10:05:45.655444 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389500 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 03 10:05:45.655444 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389503 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 03 10:05:45.655444 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389506 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 03 10:05:45.655444 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389508 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 03 10:05:45.655444 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389511 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 03 10:05:45.655444 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389513 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 03 10:05:45.655444 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389516 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 03 10:05:45.655444 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389519 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 03 10:05:45.655444 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389521 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 03 10:05:45.655444 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389524 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 03 10:05:45.655444 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389526 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 03 10:05:45.655444 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389529 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 03 10:05:45.655444 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389532 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 03 10:05:45.655444 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389535 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 03 10:05:45.655444 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389538 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 03 10:05:45.656361 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389542 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 03 10:05:45.656361 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389545 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 03 10:05:45.656361 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389550 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 03 10:05:45.656361 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389553 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 03 10:05:45.656361 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389556 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 03 10:05:45.656361 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389559 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 03 10:05:45.656361 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389562 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 03 10:05:45.656361 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389565 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 03 10:05:45.656361 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389568 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 03 10:05:45.656361 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389571 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 03 10:05:45.656361 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389574 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 03 10:05:45.656361 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389576 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 03 10:05:45.656361 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389579 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 03 10:05:45.656361 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389582 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 03 10:05:45.656361 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389584 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 03 10:05:45.656361 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389587 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 03 10:05:45.656361 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389589 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 03 10:05:45.656361 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389592 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 03 10:05:45.656361 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389595 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 03 10:05:45.656361 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389597 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 03 10:05:45.658502 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389600 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 03 10:05:45.658502 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389602 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 03 10:05:45.658502 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389605 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 03 10:05:45.658502 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389607 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 03 10:05:45.658502 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389610 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 03 10:05:45.658502 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389613 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 03 10:05:45.658502 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389615 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 03 10:05:45.658502 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389618 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 03 10:05:45.658502 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389620 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 03 10:05:45.658502 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389623 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 03 10:05:45.658502 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389626 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 03 10:05:45.658502 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389628 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 03 10:05:45.658502 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389632 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 03 10:05:45.658502 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389635 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 03 10:05:45.658502 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389638 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 03 10:05:45.658502 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389641 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 03 10:05:45.658502 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389644 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 03 10:05:45.658502 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389647 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 03 10:05:45.658502 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:45.389649 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 03 10:05:45.659409 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.389654 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 03 10:05:45.659409 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.389777 2573 server.go:962] "Client rotation is on, will bootstrap in background" Jul 03 10:05:45.659409 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.391819 2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 03 10:05:45.659409 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.393054 2573 server.go:1019] "Starting client certificate rotation" Jul 03 10:05:45.659409 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.393151 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 03 10:05:45.659409 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.393932 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 03 10:05:45.659409 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.395585 2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 03 10:05:45.659409 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.401396 2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 03 10:05:45.659409 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.412374 2573 log.go:25] "Validated CRI v1 runtime API" Jul 03 10:05:45.659409 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.417413 2573 log.go:25] "Validated CRI v1 image API" Jul 03 10:05:45.659409 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.418978 2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 03 10:05:45.659409 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.421213 2573 fs.go:135] Filesystem UUIDs: map[0773d6d0-a3d1-4a55-8392-12953efe4d58:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ebb0a836-ec1c-407e-abbe-0c340927b02f:/dev/nvme0n1p4] Jul 03 10:05:45.660282 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.421239 2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 03 10:05:45.660703 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.430649 2573 manager.go:216] Machine: {Timestamp:2026-07-03 10:05:45.430170797 +0000 UTC m=+0.342060092 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100126 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cbf1df716b6f157c48de40adb802f SystemUUID:ec2cbf1d-f716-b6f1-57c4-8de40adb802f BootID:e36ff992-abaa-4169-ac23-c0dad4ddc4d0 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1d:ff:98:1f:47 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1d:ff:98:1f:47 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:56:cb:39:5a:2a:7e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 03 10:05:45.660703 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.430763 2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 03 10:05:45.660703 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.430840 2573 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 03 10:05:45.660703 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.433008 2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 03 10:05:45.660703 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.433033 2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-191.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 03 10:05:45.660703 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.433650 2573 topology_manager.go:138] "Creating topology manager with none policy" Jul 03 10:05:45.660703 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.433661 2573 container_manager_linux.go:309] "Creating device plugin manager" Jul 03 10:05:45.660703 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.433668 2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.434246 2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.435380 2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.438115 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.438240 2573 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.438267 2573 kubelet.go:486] "Attempting to sync node with API server" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.438278 2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.438293 2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.438305 2573 kubelet.go:398] "Adding apiserver pod source" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.438316 2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.438751 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.438770 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.441067 2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.441652 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.443947 2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.443974 2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.445040 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.445054 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.445060 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.445066 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.445075 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.445080 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.445088 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.445093 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.445100 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.445105 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.445124 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.445129 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.445151 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.445157 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.448214 2573 server.go:1268] "Started kubelet" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.448279 2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.448304 2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.448962 2573 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.455835 2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.456888 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2l7j7" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.457719 2573 server.go:310] "Adding debug handlers to kubelet server" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.458007 2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-191.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:45.458530 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.458996 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2l7j7" Jul 03 10:05:45.661975 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:45.460166 2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:45.462168 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-191.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:45.462049 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-191.ec2.internal.18bebf9422fa411d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-191.ec2.internal,UID:ip-10-0-134-191.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-191.ec2.internal,},FirstTimestamp:2026-07-03 10:05:45.448186141 +0000 UTC m=+0.360075435,LastTimestamp:2026-07-03 10:05:45.448186141 +0000 UTC m=+0.360075435,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-191.ec2.internal,}" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.463542 2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.463557 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:45.463773 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.464355 2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.464364 2573 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.464378 2573 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.464432 2573 factory.go:55] Registering systemd factory Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.464447 2573 factory.go:223] Registration of the systemd container factory successfully Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.464503 2573 reconstruct.go:97] "Volume reconstruction finished" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.464514 2573 reconciler.go:29] "Reconciler: start to sync state" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.464667 2573 factory.go:153] Registering CRI-O factory Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.464683 2573 factory.go:223] Registration of the crio container factory successfully Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.464751 2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.464778 2573 factory.go:103] Registering Raw factory Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.464793 2573 manager.go:1196] Started watching for new ooms in manager Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.465206 2573 manager.go:349] Starting recovery of all containers Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.465978 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:45.468550 2573 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-134-191.ec2.internal\" not found" node="ip-10-0-134-191.ec2.internal" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.477141 2573 manager.go:354] Recovery completed Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.482266 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.488222 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientMemory" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.488251 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.488264 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientPID" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.488663 2573 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.488671 2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.488676 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.488686 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.489331 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientMemory" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.489355 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.489367 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientPID" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.490031 2573 policy_none.go:49] "None policy: Start" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.490048 2573 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 03 10:05:45.663823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.490059 2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 03 10:05:45.666141 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.491595 2573 policy_none.go:47] "Start" Jul 03 10:05:45.905144 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:45.870713 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found" Jul 03 10:05:45.905144 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.879707 2573 manager.go:337] "Starting Device Plugin manager" Jul 03 10:05:45.905144 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:45.879798 2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 03 10:05:45.905144 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.879813 2573 server.go:87] "Starting device plugin registration server" Jul 03 10:05:45.905144 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.880091 2573 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 03 10:05:45.905144 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.880103 2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 03 10:05:45.905144 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.880218 2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 03 10:05:45.905144 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.880306 2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 03 10:05:45.905144 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.880316 2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 03 10:05:45.905144 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:45.880918 2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 03 10:05:45.905144 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:45.880979 2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-191.ec2.internal\" not found" Jul 03 10:05:46.015369 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.980569 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 10:05:46.015369 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.981918 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientMemory" Jul 03 10:05:46.015369 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.981957 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 10:05:46.015369 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.981975 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientPID" Jul 03 10:05:46.015369 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.982002 2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-191.ec2.internal" Jul 03 10:05:46.015369 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:45.991855 2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-191.ec2.internal" Jul 03 10:05:46.015369 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:45.991879 2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-191.ec2.internal\": node \"ip-10-0-134-191.ec2.internal\" not found" Jul 03 10:05:46.015369 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:46.003096 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found" Jul 03 10:05:46.149721 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:46.103805 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found" Jul 03 10:05:46.265536 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:46.204075 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found" Jul 03 10:05:46.371692 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:46.304641 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found" Jul 03 10:05:46.475744 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:46.404909 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found" Jul 03 10:05:46.505185 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:46.505160 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found" Jul 03 10:05:46.507576 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.507554 2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 03 10:05:46.507653 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.507546 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-02 10:00:46 +0000 UTC" deadline="2028-02-15 01:32:19.300647734 +0000 UTC" Jul 03 10:05:46.507653 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.507597 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14199h26m32.793054474s" Jul 03 10:05:46.555185 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.555098 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 03 10:05:46.556522 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.556487 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 03 10:05:46.556522 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.556528 2573 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 03 10:05:46.556701 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.556553 2573 kubelet.go:2448] "Starting kubelet main sync loop" Jul 03 10:05:46.556701 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:46.556588 2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 03 10:05:46.559619 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.559599 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 10:05:46.607340 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:46.607302 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found" Jul 03 10:05:46.660610 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.660566 2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal"] Jul 03 10:05:46.660789 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.660660 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 10:05:46.662420 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.662402 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientMemory" Jul 03 10:05:46.662533 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.662442 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 10:05:46.662533 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.662458 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientPID" Jul 03 10:05:46.664187 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.664168 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 10:05:46.664319 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.664302 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal" Jul 03 10:05:46.664382 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.664333 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 10:05:46.665000 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.664981 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientMemory" Jul 03 10:05:46.665094 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.665015 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 10:05:46.665094 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.665029 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientPID" Jul 03 10:05:46.665094 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.664982 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientMemory" Jul 03 10:05:46.665204 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.665100 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 10:05:46.665204 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.665116 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientPID" Jul 03 10:05:46.666409 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.666394 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal" Jul 03 10:05:46.666470 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.666421 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 03 10:05:46.667201 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.667182 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientMemory" Jul 03 10:05:46.667289 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.667216 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasNoDiskPressure" Jul 03 10:05:46.667289 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.667227 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeHasSufficientPID" Jul 03 10:05:46.690571 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:46.690548 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-191.ec2.internal\" not found" node="ip-10-0-134-191.ec2.internal" Jul 03 10:05:46.695253 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:46.695236 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-191.ec2.internal\" not found" node="ip-10-0-134-191.ec2.internal" Jul 03 10:05:46.707988 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:46.707961 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found" Jul 03 10:05:46.709314 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.709296 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ce26a76a6254ac192333c41e6e1807e8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal\" (UID: \"ce26a76a6254ac192333c41e6e1807e8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal" Jul 03 10:05:46.709377 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.709323 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce26a76a6254ac192333c41e6e1807e8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal\" (UID: \"ce26a76a6254ac192333c41e6e1807e8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal" Jul 03 10:05:46.709377 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.709341 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0928399de55edb216381d354ba304559-config\") pod \"kube-apiserver-proxy-ip-10-0-134-191.ec2.internal\" (UID: \"0928399de55edb216381d354ba304559\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal" Jul 03 10:05:46.809915 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:46.809836 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-191.ec2.internal\" not found" Jul 03 10:05:46.809915 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.809892 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0928399de55edb216381d354ba304559-config\") pod \"kube-apiserver-proxy-ip-10-0-134-191.ec2.internal\" (UID: \"0928399de55edb216381d354ba304559\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal" Jul 03 10:05:46.809915 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.809915 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ce26a76a6254ac192333c41e6e1807e8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal\" (UID: \"ce26a76a6254ac192333c41e6e1807e8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal" Jul 03 10:05:46.810099 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.809933 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce26a76a6254ac192333c41e6e1807e8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal\" (UID: \"ce26a76a6254ac192333c41e6e1807e8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal" Jul 03 10:05:46.810099 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.809985 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce26a76a6254ac192333c41e6e1807e8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal\" (UID: \"ce26a76a6254ac192333c41e6e1807e8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal" Jul 03 10:05:46.810099 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.809990 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0928399de55edb216381d354ba304559-config\") pod \"kube-apiserver-proxy-ip-10-0-134-191.ec2.internal\" (UID: \"0928399de55edb216381d354ba304559\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal" Jul 03 10:05:46.810099 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.810002 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ce26a76a6254ac192333c41e6e1807e8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal\" (UID: \"ce26a76a6254ac192333c41e6e1807e8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal" Jul 03 10:05:46.820112 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.820092 2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 10:05:46.866954 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.866918 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal" Jul 03 10:05:46.880102 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.880082 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 03 10:05:46.881894 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.881880 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal" Jul 03 10:05:46.893517 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.893493 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 03 10:05:46.993244 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.993202 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal" Jul 03 10:05:46.998499 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:46.998482 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal" Jul 03 10:05:47.059489 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.059456 2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 03 10:05:47.507955 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.507930 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 03 10:05:47.508376 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.507943 2573 apiserver.go:52] "Watching apiserver" Jul 03 10:05:47.519132 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.518763 2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 03 10:05:47.519766 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:47.519738 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce26a76a6254ac192333c41e6e1807e8.slice/crio-b6c83c5dc7f5cbe5f6f79c784faf4e06834932704da6e44cd75e56fdd3064630 WatchSource:0}: Error finding container b6c83c5dc7f5cbe5f6f79c784faf4e06834932704da6e44cd75e56fdd3064630: Status 404 returned error can't find the container with id b6c83c5dc7f5cbe5f6f79c784faf4e06834932704da6e44cd75e56fdd3064630 Jul 03 10:05:47.520431 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.520407 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-qqgm8","kube-system/konnectivity-agent-s7jq9","kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz","openshift-cluster-node-tuning-operator/tuned-tr5nn","openshift-dns/node-resolver-hpxfb","openshift-multus/multus-additional-cni-plugins-62lv9","openshift-multus/network-metrics-daemon-ffkpf","openshift-image-registry/node-ca-2gnl9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal","openshift-multus/multus-tjzjw","openshift-network-diagnostics/network-check-target-kxh5b","openshift-network-operator/iptables-alerter-6bsh4","openshift-ovn-kubernetes/ovnkube-node-phvfm"] Jul 03 10:05:47.523018 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.522995 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.524324 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.524305 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 10:05:47.524595 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.524576 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6bsh4" Jul 03 10:05:47.526390 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.526372 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 03 10:05:47.526700 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.526679 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.527325 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.527295 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 03 10:05:47.527438 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.527299 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 03 10:05:47.527438 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.527375 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 03 10:05:47.528198 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.528182 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.529460 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.529440 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.530569 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.530550 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hpxfb" Jul 03 10:05:47.532443 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.532155 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2gnl9" Jul 03 10:05:47.533739 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.533721 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:05:47.533855 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:47.533804 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:05:47.535043 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.535027 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s7jq9" Jul 03 10:05:47.535888 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.535873 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 03 10:05:47.536009 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.535993 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9fbrg\"" Jul 03 10:05:47.537100 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.537082 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:05:47.537200 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:47.537152 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:05:47.538625 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.538607 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.540007 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.539970 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:05:47.540097 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:47.540044 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:05:47.542453 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.542433 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 03 10:05:47.542604 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.542586 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-b84kz\"" Jul 03 10:05:47.551858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.551829 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 03 10:05:47.552133 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.552115 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 03 10:05:47.552263 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.552234 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 03 10:05:47.552337 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.552278 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 03 10:05:47.552719 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.552703 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 03 10:05:47.553511 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.553494 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 03 10:05:47.554656 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.554634 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rt49l\"" Jul 03 10:05:47.554656 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.554649 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 03 10:05:47.554833 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.554663 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 03 10:05:47.555239 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.555218 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 03 10:05:47.555469 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.555451 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-h8jtr\"" Jul 03 10:05:47.560049 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.560006 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal" event={"ID":"ce26a76a6254ac192333c41e6e1807e8","Type":"ContainerStarted","Data":"b6c83c5dc7f5cbe5f6f79c784faf4e06834932704da6e44cd75e56fdd3064630"} Jul 03 10:05:47.566158 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.566142 2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 03 10:05:47.567762 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.567747 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 03 10:05:47.584925 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.584906 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 03 10:05:47.584925 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.584916 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-w72hs\"" Jul 03 10:05:47.585136 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.585095 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 03 10:05:47.585219 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.585118 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 03 10:05:47.585219 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.585160 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 03 10:05:47.585219 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.585189 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 03 10:05:47.585348 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.585237 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2hxjr\"" Jul 03 10:05:47.585576 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.585562 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 03 10:05:47.586231 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.586212 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 03 10:05:47.586310 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.586282 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 03 10:05:47.586365 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.586340 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 03 10:05:47.586365 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.586342 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tmv84\"" Jul 03 10:05:47.586466 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.586342 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 03 10:05:47.586466 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.586385 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4spk7\"" Jul 03 10:05:47.587279 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.587264 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lrljg\"" Jul 03 10:05:47.587279 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.587272 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 03 10:05:47.591040 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.591024 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 03 10:05:47.609943 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.609919 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-glmhf" Jul 03 10:05:47.615035 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:47.615014 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0928399de55edb216381d354ba304559.slice/crio-87b556f053934df2ae59c62d1802aba1b6d0f1c9d0a5075bdd5fa19f8e2eb794 WatchSource:0}: Error finding container 87b556f053934df2ae59c62d1802aba1b6d0f1c9d0a5075bdd5fa19f8e2eb794: Status 404 returned error can't find the container with id 87b556f053934df2ae59c62d1802aba1b6d0f1c9d0a5075bdd5fa19f8e2eb794 Jul 03 10:05:47.620862 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.620827 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-host-var-lib-cni-bin\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.620971 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.620867 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-dbus\") pod \"global-pull-secret-syncer-qqgm8\" (UID: \"e8ff375c-d36d-431c-ac6e-c2aef1d3c26a\") " pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:05:47.620971 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.620891 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret\") pod \"global-pull-secret-syncer-qqgm8\" (UID: \"e8ff375c-d36d-431c-ac6e-c2aef1d3c26a\") " pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:05:47.620971 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.620918 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-etc-kubernetes\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.620971 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.620931 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-glmhf" Jul 03 10:05:47.620971 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.620964 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-ovnkube-config\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.621213 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621010 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-os-release\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.621213 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621041 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-multus-conf-dir\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.621213 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621067 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-cni-binary-copy\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.621213 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621096 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-run-openvswitch\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.621213 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621133 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-ovnkube-script-lib\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.621213 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621172 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs\") pod \"network-metrics-daemon-ffkpf\" (UID: \"a077a8e0-9998-4603-9c7c-f92f296c2dbf\") " pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:05:47.621213 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621206 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-host-var-lib-cni-multus\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.621512 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621236 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-host-run-multus-certs\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.621512 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621260 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-run-systemd\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.621512 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621282 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7fcf8890-a36d-404f-95e1-631a6aa5c137-host\") pod \"node-ca-2gnl9\" (UID: \"7fcf8890-a36d-404f-95e1-631a6aa5c137\") " pod="openshift-image-registry/node-ca-2gnl9" Jul 03 10:05:47.621512 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621300 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.621512 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621315 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gctb4\" (UniqueName: \"kubernetes.io/projected/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-kube-api-access-gctb4\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.621512 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621332 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2pr7\" (UniqueName: \"kubernetes.io/projected/896bd6c1-8e46-4a49-ae30-aee138404618-kube-api-access-x2pr7\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.621512 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621416 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7fcf8890-a36d-404f-95e1-631a6aa5c137-serviceca\") pod \"node-ca-2gnl9\" (UID: \"7fcf8890-a36d-404f-95e1-631a6aa5c137\") " pod="openshift-image-registry/node-ca-2gnl9" Jul 03 10:05:47.621512 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621447 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76rn5\" (UniqueName: \"kubernetes.io/projected/a077a8e0-9998-4603-9c7c-f92f296c2dbf-kube-api-access-76rn5\") pod \"network-metrics-daemon-ffkpf\" (UID: \"a077a8e0-9998-4603-9c7c-f92f296c2dbf\") " pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:05:47.621512 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621474 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-cnibin\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.621512 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621502 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0238c2d7-0223-4bbd-bc30-cb8c8901f737-cni-binary-copy\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.621963 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621526 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-host-run-k8s-cni-cncf-io\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.621963 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621552 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-run-ovn-kubernetes\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.621963 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621580 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.621963 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621608 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c041b173-2b1a-40dd-9ef6-f9176e111c2e-device-dir\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.621963 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621632 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c041b173-2b1a-40dd-9ef6-f9176e111c2e-etc-selinux\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.621963 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621658 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-hostroot\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.621963 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621682 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpnjd\" (UniqueName: \"kubernetes.io/projected/0238c2d7-0223-4bbd-bc30-cb8c8901f737-kube-api-access-lpnjd\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.621963 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621734 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.621963 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621768 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-run-netns\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.621963 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621792 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-var-lib-openvswitch\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.621963 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621845 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-run-ovn\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.621963 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621875 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrvh7\" (UniqueName: \"kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7\") pod \"network-check-target-kxh5b\" (UID: \"76d0eec5-d0d7-4cf4-9229-b3a930c68566\") " pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:05:47.621963 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621897 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-kubelet\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.622399 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621923 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-cni-bin\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.622399 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621940 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c041b173-2b1a-40dd-9ef6-f9176e111c2e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.622399 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.621972 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-run\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.622399 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622011 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nksxx\" (UniqueName: \"kubernetes.io/projected/7fcf8890-a36d-404f-95e1-631a6aa5c137-kube-api-access-nksxx\") pod \"node-ca-2gnl9\" (UID: \"7fcf8890-a36d-404f-95e1-631a6aa5c137\") " pod="openshift-image-registry/node-ca-2gnl9" Jul 03 10:05:47.622399 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622048 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzfrb\" (UniqueName: \"kubernetes.io/projected/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-kube-api-access-pzfrb\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.622399 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622066 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtx6n\" (UniqueName: \"kubernetes.io/projected/1cda0625-6843-44e1-b61f-f8ad25fcea95-kube-api-access-qtx6n\") pod \"iptables-alerter-6bsh4\" (UID: \"1cda0625-6843-44e1-b61f-f8ad25fcea95\") " pod="openshift-network-operator/iptables-alerter-6bsh4" Jul 03 10:05:47.622399 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622082 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-env-overrides\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.622399 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622105 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgccd\" (UniqueName: \"kubernetes.io/projected/c041b173-2b1a-40dd-9ef6-f9176e111c2e-kube-api-access-jgccd\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.622399 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622129 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/896bd6c1-8e46-4a49-ae30-aee138404618-tmp\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.622399 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622144 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-multus-cni-dir\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.622399 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622159 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-os-release\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.622399 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622200 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c041b173-2b1a-40dd-9ef6-f9176e111c2e-sys-fs\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.622399 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622226 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-host-var-lib-kubelet\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.622858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622245 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-system-cni-dir\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.622858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622274 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-systemd-units\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.622858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622297 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-log-socket\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.622858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622312 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-lib-modules\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.622858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622327 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/896bd6c1-8e46-4a49-ae30-aee138404618-etc-tuned\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.622858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622354 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8457043a-d27e-4962-baaa-cebc19fc88dd-hosts-file\") pod \"node-resolver-hpxfb\" (UID: \"8457043a-d27e-4962-baaa-cebc19fc88dd\") " pod="openshift-dns/node-resolver-hpxfb" Jul 03 10:05:47.622858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622372 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8457043a-d27e-4962-baaa-cebc19fc88dd-tmp-dir\") pod \"node-resolver-hpxfb\" (UID: \"8457043a-d27e-4962-baaa-cebc19fc88dd\") " pod="openshift-dns/node-resolver-hpxfb" Jul 03 10:05:47.622858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622389 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-system-cni-dir\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.622858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622412 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1cda0625-6843-44e1-b61f-f8ad25fcea95-host-slash\") pod \"iptables-alerter-6bsh4\" (UID: \"1cda0625-6843-44e1-b61f-f8ad25fcea95\") " pod="openshift-network-operator/iptables-alerter-6bsh4" Jul 03 10:05:47.622858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622468 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-multus-socket-dir-parent\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.622858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622493 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-etc-kubernetes\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.622858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622523 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c041b173-2b1a-40dd-9ef6-f9176e111c2e-registration-dir\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.622858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622542 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-var-lib-kubelet\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.622858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622563 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-host\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.623310 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622585 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3c7a6ee8-12fe-4c0e-a0b2-e3b828f94858-agent-certs\") pod \"konnectivity-agent-s7jq9\" (UID: \"3c7a6ee8-12fe-4c0e-a0b2-e3b828f94858\") " pod="kube-system/konnectivity-agent-s7jq9" Jul 03 10:05:47.623310 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622601 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-kubelet-config\") pod \"global-pull-secret-syncer-qqgm8\" (UID: \"e8ff375c-d36d-431c-ac6e-c2aef1d3c26a\") " pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:05:47.623310 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622617 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c041b173-2b1a-40dd-9ef6-f9176e111c2e-socket-dir\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.623310 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622633 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g749n\" (UniqueName: \"kubernetes.io/projected/8457043a-d27e-4962-baaa-cebc19fc88dd-kube-api-access-g749n\") pod \"node-resolver-hpxfb\" (UID: \"8457043a-d27e-4962-baaa-cebc19fc88dd\") " pod="openshift-dns/node-resolver-hpxfb" Jul 03 10:05:47.623310 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622660 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-cnibin\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.623310 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622690 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.623310 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622708 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1cda0625-6843-44e1-b61f-f8ad25fcea95-iptables-alerter-script\") pod \"iptables-alerter-6bsh4\" (UID: \"1cda0625-6843-44e1-b61f-f8ad25fcea95\") " pod="openshift-network-operator/iptables-alerter-6bsh4" Jul 03 10:05:47.623310 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622724 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-slash\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.623310 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622758 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-cni-netd\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.623310 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622786 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-etc-modprobe-d\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.623310 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622821 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-etc-systemd\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.623310 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622857 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3c7a6ee8-12fe-4c0e-a0b2-e3b828f94858-konnectivity-ca\") pod \"konnectivity-agent-s7jq9\" (UID: \"3c7a6ee8-12fe-4c0e-a0b2-e3b828f94858\") " pod="kube-system/konnectivity-agent-s7jq9" Jul 03 10:05:47.623310 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622876 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-node-log\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.623743 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622899 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-etc-sysconfig\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.623743 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622916 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-etc-sysctl-d\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.623743 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622932 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-etc-sysctl-conf\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.623743 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622948 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-sys\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.623743 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622968 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-host-run-netns\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.623743 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.622983 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0238c2d7-0223-4bbd-bc30-cb8c8901f737-multus-daemon-config\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.623743 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.623002 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-etc-openvswitch\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.623743 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.623019 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-ovn-node-metrics-cert\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.723404 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723315 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-host-var-lib-kubelet\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.723404 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723348 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-system-cni-dir\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.723404 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723368 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-systemd-units\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.723404 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723383 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-log-socket\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.723404 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723403 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-lib-modules\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.723788 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723428 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/896bd6c1-8e46-4a49-ae30-aee138404618-etc-tuned\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.723788 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723428 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-host-var-lib-kubelet\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.723788 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723447 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-system-cni-dir\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.723788 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723459 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-systemd-units\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.723788 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723454 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8457043a-d27e-4962-baaa-cebc19fc88dd-hosts-file\") pod \"node-resolver-hpxfb\" (UID: \"8457043a-d27e-4962-baaa-cebc19fc88dd\") " pod="openshift-dns/node-resolver-hpxfb" Jul 03 10:05:47.723788 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723480 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-log-socket\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.723788 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723494 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8457043a-d27e-4962-baaa-cebc19fc88dd-tmp-dir\") pod \"node-resolver-hpxfb\" (UID: \"8457043a-d27e-4962-baaa-cebc19fc88dd\") " pod="openshift-dns/node-resolver-hpxfb" Jul 03 10:05:47.723788 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723528 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-system-cni-dir\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.723788 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723550 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8457043a-d27e-4962-baaa-cebc19fc88dd-hosts-file\") pod \"node-resolver-hpxfb\" (UID: \"8457043a-d27e-4962-baaa-cebc19fc88dd\") " pod="openshift-dns/node-resolver-hpxfb" Jul 03 10:05:47.723788 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723561 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1cda0625-6843-44e1-b61f-f8ad25fcea95-host-slash\") pod \"iptables-alerter-6bsh4\" (UID: \"1cda0625-6843-44e1-b61f-f8ad25fcea95\") " pod="openshift-network-operator/iptables-alerter-6bsh4" Jul 03 10:05:47.723788 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723585 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-multus-socket-dir-parent\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.723788 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723589 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-system-cni-dir\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.723788 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723598 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-lib-modules\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.723788 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723610 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-etc-kubernetes\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.723788 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723637 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c041b173-2b1a-40dd-9ef6-f9176e111c2e-registration-dir\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.724459 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723661 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-var-lib-kubelet\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.724459 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723680 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c041b173-2b1a-40dd-9ef6-f9176e111c2e-registration-dir\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.724459 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723686 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-host\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.724459 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723681 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-etc-kubernetes\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.724459 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723639 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1cda0625-6843-44e1-b61f-f8ad25fcea95-host-slash\") pod \"iptables-alerter-6bsh4\" (UID: \"1cda0625-6843-44e1-b61f-f8ad25fcea95\") " pod="openshift-network-operator/iptables-alerter-6bsh4" Jul 03 10:05:47.724459 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723712 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3c7a6ee8-12fe-4c0e-a0b2-e3b828f94858-agent-certs\") pod \"konnectivity-agent-s7jq9\" (UID: \"3c7a6ee8-12fe-4c0e-a0b2-e3b828f94858\") " pod="kube-system/konnectivity-agent-s7jq9" Jul 03 10:05:47.724459 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723730 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-multus-socket-dir-parent\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.724459 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723766 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-var-lib-kubelet\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.724459 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723784 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-host\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.724459 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723810 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-kubelet-config\") pod \"global-pull-secret-syncer-qqgm8\" (UID: \"e8ff375c-d36d-431c-ac6e-c2aef1d3c26a\") " pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:05:47.724459 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723853 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c041b173-2b1a-40dd-9ef6-f9176e111c2e-socket-dir\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.724459 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723849 2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 03 10:05:47.724459 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723882 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8457043a-d27e-4962-baaa-cebc19fc88dd-tmp-dir\") pod \"node-resolver-hpxfb\" (UID: \"8457043a-d27e-4962-baaa-cebc19fc88dd\") " pod="openshift-dns/node-resolver-hpxfb" Jul 03 10:05:47.724459 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723905 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-kubelet-config\") pod \"global-pull-secret-syncer-qqgm8\" (UID: \"e8ff375c-d36d-431c-ac6e-c2aef1d3c26a\") " pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:05:47.724459 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723884 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g749n\" (UniqueName: \"kubernetes.io/projected/8457043a-d27e-4962-baaa-cebc19fc88dd-kube-api-access-g749n\") pod \"node-resolver-hpxfb\" (UID: \"8457043a-d27e-4962-baaa-cebc19fc88dd\") " pod="openshift-dns/node-resolver-hpxfb" Jul 03 10:05:47.725169 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723963 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-cnibin\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.725169 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.723986 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c041b173-2b1a-40dd-9ef6-f9176e111c2e-socket-dir\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.725169 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.724165 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.725169 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.724346 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1cda0625-6843-44e1-b61f-f8ad25fcea95-iptables-alerter-script\") pod \"iptables-alerter-6bsh4\" (UID: \"1cda0625-6843-44e1-b61f-f8ad25fcea95\") " pod="openshift-network-operator/iptables-alerter-6bsh4" Jul 03 10:05:47.725169 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.724388 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-slash\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.725169 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.724462 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-cnibin\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.725169 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.724639 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-slash\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.725169 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725154 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-cni-netd\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.725556 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725200 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-etc-modprobe-d\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.725556 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725234 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-etc-systemd\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.725556 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725272 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3c7a6ee8-12fe-4c0e-a0b2-e3b828f94858-konnectivity-ca\") pod \"konnectivity-agent-s7jq9\" (UID: \"3c7a6ee8-12fe-4c0e-a0b2-e3b828f94858\") " pod="kube-system/konnectivity-agent-s7jq9" Jul 03 10:05:47.725556 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725299 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-node-log\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.725556 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725332 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-etc-sysconfig\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.725556 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725365 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-etc-sysctl-d\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.725556 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725397 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-etc-sysctl-conf\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.725556 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725430 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-sys\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.725556 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725458 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-host-run-netns\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.725556 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725493 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0238c2d7-0223-4bbd-bc30-cb8c8901f737-multus-daemon-config\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.725556 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725525 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-etc-openvswitch\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.726081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725562 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-ovn-node-metrics-cert\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.726081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725594 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-host-var-lib-cni-bin\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.726081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725674 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-node-log\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.726081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725715 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-etc-sysctl-d\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.726081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725743 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-etc-sysconfig\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.726081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725796 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-host-var-lib-cni-bin\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.726081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725821 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-sys\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.726081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725839 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-host-run-netns\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.726081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725947 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-etc-sysctl-conf\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.726081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.725991 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-dbus\") pod \"global-pull-secret-syncer-qqgm8\" (UID: \"e8ff375c-d36d-431c-ac6e-c2aef1d3c26a\") " pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:05:47.726081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.726031 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret\") pod \"global-pull-secret-syncer-qqgm8\" (UID: \"e8ff375c-d36d-431c-ac6e-c2aef1d3c26a\") " pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:05:47.726587 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.726070 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-etc-kubernetes\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.726587 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.726125 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-ovnkube-config\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.726587 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.726160 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-os-release\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.726587 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.726189 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-multus-conf-dir\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.726587 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.726223 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-cni-binary-copy\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.726587 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.726260 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-run-openvswitch\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.726587 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.726297 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-ovnkube-script-lib\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.726587 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.726325 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs\") pod \"network-metrics-daemon-ffkpf\" (UID: \"a077a8e0-9998-4603-9c7c-f92f296c2dbf\") " pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:05:47.726587 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.726361 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-host-var-lib-cni-multus\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.726587 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.726395 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-host-run-multus-certs\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.726587 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.726406 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0238c2d7-0223-4bbd-bc30-cb8c8901f737-multus-daemon-config\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.726587 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.726427 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-run-systemd\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.726587 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.726468 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3c7a6ee8-12fe-4c0e-a0b2-e3b828f94858-konnectivity-ca\") pod \"konnectivity-agent-s7jq9\" (UID: \"3c7a6ee8-12fe-4c0e-a0b2-e3b828f94858\") " pod="kube-system/konnectivity-agent-s7jq9" Jul 03 10:05:47.726587 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.726480 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-run-systemd\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.726587 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.726485 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7fcf8890-a36d-404f-95e1-631a6aa5c137-host\") pod \"node-ca-2gnl9\" (UID: \"7fcf8890-a36d-404f-95e1-631a6aa5c137\") " pod="openshift-image-registry/node-ca-2gnl9" Jul 03 10:05:47.727305 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.726552 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-os-release\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.727305 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.726575 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-cni-netd\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.727305 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.726596 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.727305 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:47.726714 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:47.727305 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:47.726852 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs podName:a077a8e0-9998-4603-9c7c-f92f296c2dbf nodeName:}" failed. No retries permitted until 2026-07-03 10:05:48.226779403 +0000 UTC m=+3.138668711 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs") pod "network-metrics-daemon-ffkpf" (UID: "a077a8e0-9998-4603-9c7c-f92f296c2dbf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:47.727305 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.727095 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-ovnkube-script-lib\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.727305 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.727155 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1cda0625-6843-44e1-b61f-f8ad25fcea95-iptables-alerter-script\") pod \"iptables-alerter-6bsh4\" (UID: \"1cda0625-6843-44e1-b61f-f8ad25fcea95\") " pod="openshift-network-operator/iptables-alerter-6bsh4" Jul 03 10:05:47.727305 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.727156 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-run-openvswitch\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.727305 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.727205 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-etc-systemd\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.727305 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.727232 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-host-var-lib-cni-multus\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.727835 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.727716 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-dbus\") pod \"global-pull-secret-syncer-qqgm8\" (UID: \"e8ff375c-d36d-431c-ac6e-c2aef1d3c26a\") " pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:05:47.727835 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.727733 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.727835 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.727733 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-cni-binary-copy\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.727835 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.727820 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gctb4\" (UniqueName: \"kubernetes.io/projected/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-kube-api-access-gctb4\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.728111 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.727858 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2pr7\" (UniqueName: \"kubernetes.io/projected/896bd6c1-8e46-4a49-ae30-aee138404618-kube-api-access-x2pr7\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.728111 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.727889 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7fcf8890-a36d-404f-95e1-631a6aa5c137-serviceca\") pod \"node-ca-2gnl9\" (UID: \"7fcf8890-a36d-404f-95e1-631a6aa5c137\") " pod="openshift-image-registry/node-ca-2gnl9" Jul 03 10:05:47.728111 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.727921 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76rn5\" (UniqueName: \"kubernetes.io/projected/a077a8e0-9998-4603-9c7c-f92f296c2dbf-kube-api-access-76rn5\") pod \"network-metrics-daemon-ffkpf\" (UID: \"a077a8e0-9998-4603-9c7c-f92f296c2dbf\") " pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:05:47.728570 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.728359 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-cnibin\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.728570 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.728397 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7fcf8890-a36d-404f-95e1-631a6aa5c137-serviceca\") pod \"node-ca-2gnl9\" (UID: \"7fcf8890-a36d-404f-95e1-631a6aa5c137\") " pod="openshift-image-registry/node-ca-2gnl9" Jul 03 10:05:47.728570 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.728396 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/896bd6c1-8e46-4a49-ae30-aee138404618-etc-tuned\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.728570 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.728412 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0238c2d7-0223-4bbd-bc30-cb8c8901f737-cni-binary-copy\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.728570 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.728477 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-etc-kubernetes\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.728570 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.728533 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7fcf8890-a36d-404f-95e1-631a6aa5c137-host\") pod \"node-ca-2gnl9\" (UID: \"7fcf8890-a36d-404f-95e1-631a6aa5c137\") " pod="openshift-image-registry/node-ca-2gnl9" Jul 03 10:05:47.729016 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.728668 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-host-run-multus-certs\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.729016 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.728683 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.729016 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.728782 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-etc-modprobe-d\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.729016 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:47.728873 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:47.729016 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:47.728939 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret podName:e8ff375c-d36d-431c-ac6e-c2aef1d3c26a nodeName:}" failed. No retries permitted until 2026-07-03 10:05:48.228920444 +0000 UTC m=+3.140809742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret") pod "global-pull-secret-syncer-qqgm8" (UID: "e8ff375c-d36d-431c-ac6e-c2aef1d3c26a") : object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:47.729016 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.728996 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-multus-conf-dir\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.729311 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729044 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3c7a6ee8-12fe-4c0e-a0b2-e3b828f94858-agent-certs\") pod \"konnectivity-agent-s7jq9\" (UID: \"3c7a6ee8-12fe-4c0e-a0b2-e3b828f94858\") " pod="kube-system/konnectivity-agent-s7jq9" Jul 03 10:05:47.729311 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729281 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-host-run-k8s-cni-cncf-io\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.729406 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729323 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-run-ovn-kubernetes\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.729406 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729355 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.729406 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729388 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c041b173-2b1a-40dd-9ef6-f9176e111c2e-device-dir\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.729557 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729417 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c041b173-2b1a-40dd-9ef6-f9176e111c2e-etc-selinux\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.729557 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729441 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-hostroot\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.729557 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729472 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpnjd\" (UniqueName: \"kubernetes.io/projected/0238c2d7-0223-4bbd-bc30-cb8c8901f737-kube-api-access-lpnjd\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.729557 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729504 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.729557 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729535 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-run-netns\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.729883 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729562 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-var-lib-openvswitch\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.729883 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729592 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-run-ovn\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.729883 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729623 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrvh7\" (UniqueName: \"kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7\") pod \"network-check-target-kxh5b\" (UID: \"76d0eec5-d0d7-4cf4-9229-b3a930c68566\") " pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:05:47.729883 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729653 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-kubelet\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.729883 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729682 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-cni-bin\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.729883 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729658 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-ovnkube-config\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.729883 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729710 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c041b173-2b1a-40dd-9ef6-f9176e111c2e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.729883 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729749 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-run-ovn-kubernetes\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.729883 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729773 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-kubelet\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.729883 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729783 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c041b173-2b1a-40dd-9ef6-f9176e111c2e-device-dir\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.729883 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729882 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-cni-bin\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.730518 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729905 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-run-ovn\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.730518 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729953 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-host-run-k8s-cni-cncf-io\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.730518 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729987 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c041b173-2b1a-40dd-9ef6-f9176e111c2e-etc-selinux\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.730518 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.729786 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-run\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.730518 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.730064 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nksxx\" (UniqueName: \"kubernetes.io/projected/7fcf8890-a36d-404f-95e1-631a6aa5c137-kube-api-access-nksxx\") pod \"node-ca-2gnl9\" (UID: \"7fcf8890-a36d-404f-95e1-631a6aa5c137\") " pod="openshift-image-registry/node-ca-2gnl9" Jul 03 10:05:47.730518 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.730215 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c041b173-2b1a-40dd-9ef6-f9176e111c2e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.730518 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.730502 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-var-lib-openvswitch\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.730916 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.730821 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-run-netns\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.730916 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.730828 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-etc-openvswitch\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.730916 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.730826 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzfrb\" (UniqueName: \"kubernetes.io/projected/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-kube-api-access-pzfrb\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.730916 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.730892 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtx6n\" (UniqueName: \"kubernetes.io/projected/1cda0625-6843-44e1-b61f-f8ad25fcea95-kube-api-access-qtx6n\") pod \"iptables-alerter-6bsh4\" (UID: \"1cda0625-6843-44e1-b61f-f8ad25fcea95\") " pod="openshift-network-operator/iptables-alerter-6bsh4" Jul 03 10:05:47.731091 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.730913 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-hostroot\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.731091 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.730936 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/896bd6c1-8e46-4a49-ae30-aee138404618-run\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.731091 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.730975 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0238c2d7-0223-4bbd-bc30-cb8c8901f737-cni-binary-copy\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.731091 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.730935 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-env-overrides\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.731091 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.730919 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-ovn-node-metrics-cert\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.731499 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.731461 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.731499 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.731466 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgccd\" (UniqueName: \"kubernetes.io/projected/c041b173-2b1a-40dd-9ef6-f9176e111c2e-kube-api-access-jgccd\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.731636 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.731517 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/896bd6c1-8e46-4a49-ae30-aee138404618-tmp\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.731636 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.731534 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-cnibin\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.731636 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.731575 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-multus-cni-dir\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.731636 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.731613 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-os-release\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.731888 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.731645 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c041b173-2b1a-40dd-9ef6-f9176e111c2e-sys-fs\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.731888 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.731701 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0238c2d7-0223-4bbd-bc30-cb8c8901f737-multus-cni-dir\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.731888 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.731732 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c041b173-2b1a-40dd-9ef6-f9176e111c2e-sys-fs\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.731888 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.731825 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-os-release\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.731888 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.731825 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.732094 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.732071 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-env-overrides\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.733430 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.733405 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/896bd6c1-8e46-4a49-ae30-aee138404618-tmp\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.734006 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.733981 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g749n\" (UniqueName: \"kubernetes.io/projected/8457043a-d27e-4962-baaa-cebc19fc88dd-kube-api-access-g749n\") pod \"node-resolver-hpxfb\" (UID: \"8457043a-d27e-4962-baaa-cebc19fc88dd\") " pod="openshift-dns/node-resolver-hpxfb" Jul 03 10:05:47.744591 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.744571 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76rn5\" (UniqueName: \"kubernetes.io/projected/a077a8e0-9998-4603-9c7c-f92f296c2dbf-kube-api-access-76rn5\") pod \"network-metrics-daemon-ffkpf\" (UID: \"a077a8e0-9998-4603-9c7c-f92f296c2dbf\") " pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:05:47.748673 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:47.748653 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 10:05:47.748811 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:47.748785 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 10:05:47.748916 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:47.748825 2573 projected.go:196] Error preparing data for projected volume kube-api-access-qrvh7 for pod openshift-network-diagnostics/network-check-target-kxh5b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:47.748916 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:47.748891 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7 podName:76d0eec5-d0d7-4cf4-9229-b3a930c68566 nodeName:}" failed. No retries permitted until 2026-07-03 10:05:48.24887197 +0000 UTC m=+3.160761254 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qrvh7" (UniqueName: "kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7") pod "network-check-target-kxh5b" (UID: "76d0eec5-d0d7-4cf4-9229-b3a930c68566") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:47.750614 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.750586 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpnjd\" (UniqueName: \"kubernetes.io/projected/0238c2d7-0223-4bbd-bc30-cb8c8901f737-kube-api-access-lpnjd\") pod \"multus-tjzjw\" (UID: \"0238c2d7-0223-4bbd-bc30-cb8c8901f737\") " pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.750614 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.750601 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzfrb\" (UniqueName: \"kubernetes.io/projected/4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e-kube-api-access-pzfrb\") pod \"multus-additional-cni-plugins-62lv9\" (UID: \"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e\") " pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.750857 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.750827 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtx6n\" (UniqueName: \"kubernetes.io/projected/1cda0625-6843-44e1-b61f-f8ad25fcea95-kube-api-access-qtx6n\") pod \"iptables-alerter-6bsh4\" (UID: \"1cda0625-6843-44e1-b61f-f8ad25fcea95\") " pod="openshift-network-operator/iptables-alerter-6bsh4" Jul 03 10:05:47.751295 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.751275 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2pr7\" (UniqueName: \"kubernetes.io/projected/896bd6c1-8e46-4a49-ae30-aee138404618-kube-api-access-x2pr7\") pod \"tuned-tr5nn\" (UID: \"896bd6c1-8e46-4a49-ae30-aee138404618\") " pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.751403 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.751385 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nksxx\" (UniqueName: \"kubernetes.io/projected/7fcf8890-a36d-404f-95e1-631a6aa5c137-kube-api-access-nksxx\") pod \"node-ca-2gnl9\" (UID: \"7fcf8890-a36d-404f-95e1-631a6aa5c137\") " pod="openshift-image-registry/node-ca-2gnl9" Jul 03 10:05:47.751785 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.751763 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gctb4\" (UniqueName: \"kubernetes.io/projected/ff752ee3-d640-40b6-bd9a-6532dbe9dfc0-kube-api-access-gctb4\") pod \"ovnkube-node-phvfm\" (UID: \"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0\") " pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.751882 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.751806 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgccd\" (UniqueName: \"kubernetes.io/projected/c041b173-2b1a-40dd-9ef6-f9176e111c2e-kube-api-access-jgccd\") pod \"aws-ebs-csi-driver-node-6vlkz\" (UID: \"c041b173-2b1a-40dd-9ef6-f9176e111c2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.834802 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.834774 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-62lv9" Jul 03 10:05:47.841273 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.841247 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6bsh4" Jul 03 10:05:47.843664 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:47.843644 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b8cd90a_dd75_4ddc_b0cb_a4d4573bd38e.slice/crio-b594b2648df42c41f7733c394d2cf7f50dc2ecce63afd6cc8f4fbee78688f365 WatchSource:0}: Error finding container b594b2648df42c41f7733c394d2cf7f50dc2ecce63afd6cc8f4fbee78688f365: Status 404 returned error can't find the container with id b594b2648df42c41f7733c394d2cf7f50dc2ecce63afd6cc8f4fbee78688f365 Jul 03 10:05:47.848356 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.848279 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:05:47.849739 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:47.849715 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cda0625_6843_44e1_b61f_f8ad25fcea95.slice/crio-d3096dc4d14e7d63859b75f5d51df1de2712bd0c6538348111b0643dbddc3371 WatchSource:0}: Error finding container d3096dc4d14e7d63859b75f5d51df1de2712bd0c6538348111b0643dbddc3371: Status 404 returned error can't find the container with id d3096dc4d14e7d63859b75f5d51df1de2712bd0c6538348111b0643dbddc3371 Jul 03 10:05:47.852895 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.852877 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" Jul 03 10:05:47.856849 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:47.856816 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff752ee3_d640_40b6_bd9a_6532dbe9dfc0.slice/crio-10d7798ccd9317960608dc70115c7f7ea9b46603ba27b71f3f45f800fa298d01 WatchSource:0}: Error finding container 10d7798ccd9317960608dc70115c7f7ea9b46603ba27b71f3f45f800fa298d01: Status 404 returned error can't find the container with id 10d7798ccd9317960608dc70115c7f7ea9b46603ba27b71f3f45f800fa298d01 Jul 03 10:05:47.861261 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:47.861241 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc041b173_2b1a_40dd_9ef6_f9176e111c2e.slice/crio-e2515358f3fac778ebabd8cf3e512315de36f256c926af1b55f9cd7ec941a155 WatchSource:0}: Error finding container e2515358f3fac778ebabd8cf3e512315de36f256c926af1b55f9cd7ec941a155: Status 404 returned error can't find the container with id e2515358f3fac778ebabd8cf3e512315de36f256c926af1b55f9cd7ec941a155 Jul 03 10:05:47.861912 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.861895 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" Jul 03 10:05:47.868893 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.868871 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hpxfb" Jul 03 10:05:47.869627 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:47.869611 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod896bd6c1_8e46_4a49_ae30_aee138404618.slice/crio-7b6a4ad5159c2e9baab7891687e6399b8820a9e774ae3441b6601781601396cc WatchSource:0}: Error finding container 7b6a4ad5159c2e9baab7891687e6399b8820a9e774ae3441b6601781601396cc: Status 404 returned error can't find the container with id 7b6a4ad5159c2e9baab7891687e6399b8820a9e774ae3441b6601781601396cc Jul 03 10:05:47.873891 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.873875 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2gnl9" Jul 03 10:05:47.877347 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:47.877327 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8457043a_d27e_4962_baaa_cebc19fc88dd.slice/crio-bca517c46583979ac4205ba7110b3a24787354db658e06ee5e2ea42aa402f9ae WatchSource:0}: Error finding container bca517c46583979ac4205ba7110b3a24787354db658e06ee5e2ea42aa402f9ae: Status 404 returned error can't find the container with id bca517c46583979ac4205ba7110b3a24787354db658e06ee5e2ea42aa402f9ae Jul 03 10:05:47.880075 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.880055 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s7jq9" Jul 03 10:05:47.882473 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:47.882443 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fcf8890_a36d_404f_95e1_631a6aa5c137.slice/crio-9a39f0c2828a470c66345ef5f3b497f866b130bc5c93f340a1bcd3910ab70e7d WatchSource:0}: Error finding container 9a39f0c2828a470c66345ef5f3b497f866b130bc5c93f340a1bcd3910ab70e7d: Status 404 returned error can't find the container with id 9a39f0c2828a470c66345ef5f3b497f866b130bc5c93f340a1bcd3910ab70e7d Jul 03 10:05:47.884877 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:47.884802 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tjzjw" Jul 03 10:05:47.890903 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:47.890881 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c7a6ee8_12fe_4c0e_a0b2_e3b828f94858.slice/crio-58b11f7b66144619c18c7ee50b25ffe0a10243e86deefb40490ee844a8e934a0 WatchSource:0}: Error finding container 58b11f7b66144619c18c7ee50b25ffe0a10243e86deefb40490ee844a8e934a0: Status 404 returned error can't find the container with id 58b11f7b66144619c18c7ee50b25ffe0a10243e86deefb40490ee844a8e934a0 Jul 03 10:05:47.893668 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:05:47.893643 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0238c2d7_0223_4bbd_bc30_cb8c8901f737.slice/crio-990b51b4443f54c5e4f62eb5427e8b6c02a0bd3659ad59975d14a6b3ab5e7eaf WatchSource:0}: Error finding container 990b51b4443f54c5e4f62eb5427e8b6c02a0bd3659ad59975d14a6b3ab5e7eaf: Status 404 returned error can't find the container with id 990b51b4443f54c5e4f62eb5427e8b6c02a0bd3659ad59975d14a6b3ab5e7eaf Jul 03 10:05:48.238287 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:48.238248 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret\") pod \"global-pull-secret-syncer-qqgm8\" (UID: \"e8ff375c-d36d-431c-ac6e-c2aef1d3c26a\") " pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:05:48.238473 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:48.238302 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs\") pod \"network-metrics-daemon-ffkpf\" (UID: \"a077a8e0-9998-4603-9c7c-f92f296c2dbf\") " pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:05:48.238473 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:48.238426 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:48.238616 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:48.238503 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret podName:e8ff375c-d36d-431c-ac6e-c2aef1d3c26a nodeName:}" failed. No retries permitted until 2026-07-03 10:05:49.238478572 +0000 UTC m=+4.150367881 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret") pod "global-pull-secret-syncer-qqgm8" (UID: "e8ff375c-d36d-431c-ac6e-c2aef1d3c26a") : object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:48.238616 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:48.238551 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:48.238616 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:48.238606 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs podName:a077a8e0-9998-4603-9c7c-f92f296c2dbf nodeName:}" failed. No retries permitted until 2026-07-03 10:05:49.238588165 +0000 UTC m=+4.150477453 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs") pod "network-metrics-daemon-ffkpf" (UID: "a077a8e0-9998-4603-9c7c-f92f296c2dbf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:48.339891 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:48.338980 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrvh7\" (UniqueName: \"kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7\") pod \"network-check-target-kxh5b\" (UID: \"76d0eec5-d0d7-4cf4-9229-b3a930c68566\") " pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:05:48.339891 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:48.339264 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 10:05:48.339891 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:48.339292 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 10:05:48.339891 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:48.339313 2573 projected.go:196] Error preparing data for projected volume kube-api-access-qrvh7 for pod openshift-network-diagnostics/network-check-target-kxh5b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:48.339891 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:48.339380 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7 podName:76d0eec5-d0d7-4cf4-9229-b3a930c68566 nodeName:}" failed. No retries permitted until 2026-07-03 10:05:49.339357113 +0000 UTC m=+4.251246422 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qrvh7" (UniqueName: "kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7") pod "network-check-target-kxh5b" (UID: "76d0eec5-d0d7-4cf4-9229-b3a930c68566") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:48.582266 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:48.582164 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2gnl9" event={"ID":"7fcf8890-a36d-404f-95e1-631a6aa5c137","Type":"ContainerStarted","Data":"9a39f0c2828a470c66345ef5f3b497f866b130bc5c93f340a1bcd3910ab70e7d"} Jul 03 10:05:48.584784 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:48.584750 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hpxfb" event={"ID":"8457043a-d27e-4962-baaa-cebc19fc88dd","Type":"ContainerStarted","Data":"bca517c46583979ac4205ba7110b3a24787354db658e06ee5e2ea42aa402f9ae"} Jul 03 10:05:48.609069 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:48.609023 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" event={"ID":"896bd6c1-8e46-4a49-ae30-aee138404618","Type":"ContainerStarted","Data":"7b6a4ad5159c2e9baab7891687e6399b8820a9e774ae3441b6601781601396cc"} Jul 03 10:05:48.624429 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:48.624141 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-02 10:00:48 +0000 UTC" deadline="2027-12-24 20:19:09.318440183 +0000 UTC" Jul 03 10:05:48.624429 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:48.624178 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12946h13m20.694267203s" Jul 03 10:05:48.626338 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:48.626307 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" event={"ID":"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0","Type":"ContainerStarted","Data":"10d7798ccd9317960608dc70115c7f7ea9b46603ba27b71f3f45f800fa298d01"} Jul 03 10:05:48.631240 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:48.631211 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal" event={"ID":"0928399de55edb216381d354ba304559","Type":"ContainerStarted","Data":"87b556f053934df2ae59c62d1802aba1b6d0f1c9d0a5075bdd5fa19f8e2eb794"} Jul 03 10:05:48.664708 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:48.663833 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s7jq9" event={"ID":"3c7a6ee8-12fe-4c0e-a0b2-e3b828f94858","Type":"ContainerStarted","Data":"58b11f7b66144619c18c7ee50b25ffe0a10243e86deefb40490ee844a8e934a0"} Jul 03 10:05:48.687524 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:48.687482 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" event={"ID":"c041b173-2b1a-40dd-9ef6-f9176e111c2e","Type":"ContainerStarted","Data":"e2515358f3fac778ebabd8cf3e512315de36f256c926af1b55f9cd7ec941a155"} Jul 03 10:05:48.716704 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:48.716593 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6bsh4" event={"ID":"1cda0625-6843-44e1-b61f-f8ad25fcea95","Type":"ContainerStarted","Data":"d3096dc4d14e7d63859b75f5d51df1de2712bd0c6538348111b0643dbddc3371"} Jul 03 10:05:48.726164 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:48.726081 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62lv9" event={"ID":"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e","Type":"ContainerStarted","Data":"b594b2648df42c41f7733c394d2cf7f50dc2ecce63afd6cc8f4fbee78688f365"} Jul 03 10:05:48.750574 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:48.750528 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tjzjw" event={"ID":"0238c2d7-0223-4bbd-bc30-cb8c8901f737","Type":"ContainerStarted","Data":"990b51b4443f54c5e4f62eb5427e8b6c02a0bd3659ad59975d14a6b3ab5e7eaf"} Jul 03 10:05:49.247332 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:49.247294 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret\") pod \"global-pull-secret-syncer-qqgm8\" (UID: \"e8ff375c-d36d-431c-ac6e-c2aef1d3c26a\") " pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:05:49.247542 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:49.247351 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs\") pod \"network-metrics-daemon-ffkpf\" (UID: \"a077a8e0-9998-4603-9c7c-f92f296c2dbf\") " pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:05:49.247542 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:49.247482 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:49.247542 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:49.247484 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:49.247723 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:49.247550 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs podName:a077a8e0-9998-4603-9c7c-f92f296c2dbf nodeName:}" failed. No retries permitted until 2026-07-03 10:05:51.247529087 +0000 UTC m=+6.159418390 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs") pod "network-metrics-daemon-ffkpf" (UID: "a077a8e0-9998-4603-9c7c-f92f296c2dbf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:49.247723 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:49.247571 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret podName:e8ff375c-d36d-431c-ac6e-c2aef1d3c26a nodeName:}" failed. No retries permitted until 2026-07-03 10:05:51.247559966 +0000 UTC m=+6.159449256 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret") pod "global-pull-secret-syncer-qqgm8" (UID: "e8ff375c-d36d-431c-ac6e-c2aef1d3c26a") : object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:49.348365 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:49.348326 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrvh7\" (UniqueName: \"kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7\") pod \"network-check-target-kxh5b\" (UID: \"76d0eec5-d0d7-4cf4-9229-b3a930c68566\") " pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:05:49.348556 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:49.348519 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 10:05:49.348556 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:49.348542 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 10:05:49.348556 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:49.348554 2573 projected.go:196] Error preparing data for projected volume kube-api-access-qrvh7 for pod openshift-network-diagnostics/network-check-target-kxh5b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:49.348739 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:49.348615 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7 podName:76d0eec5-d0d7-4cf4-9229-b3a930c68566 nodeName:}" failed. No retries permitted until 2026-07-03 10:05:51.348593572 +0000 UTC m=+6.260482879 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qrvh7" (UniqueName: "kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7") pod "network-check-target-kxh5b" (UID: "76d0eec5-d0d7-4cf4-9229-b3a930c68566") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:49.557110 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:49.557028 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:05:49.557110 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:49.557065 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:05:49.557341 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:49.557176 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:05:49.557341 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:49.557243 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:05:49.557341 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:49.557287 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:05:49.557514 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:49.557366 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:05:49.625418 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:49.625344 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-02 10:00:48 +0000 UTC" deadline="2028-03-16 13:08:28.181164042 +0000 UTC" Jul 03 10:05:49.625418 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:49.625417 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14931h2m38.555751332s" Jul 03 10:05:51.265311 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:51.265273 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs\") pod \"network-metrics-daemon-ffkpf\" (UID: \"a077a8e0-9998-4603-9c7c-f92f296c2dbf\") " pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:05:51.265969 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:51.265385 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret\") pod \"global-pull-secret-syncer-qqgm8\" (UID: \"e8ff375c-d36d-431c-ac6e-c2aef1d3c26a\") " pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:05:51.265969 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:51.265475 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:51.265969 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:51.265509 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:51.265969 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:51.265563 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs podName:a077a8e0-9998-4603-9c7c-f92f296c2dbf nodeName:}" failed. No retries permitted until 2026-07-03 10:05:55.265539887 +0000 UTC m=+10.177429186 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs") pod "network-metrics-daemon-ffkpf" (UID: "a077a8e0-9998-4603-9c7c-f92f296c2dbf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:51.265969 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:51.265586 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret podName:e8ff375c-d36d-431c-ac6e-c2aef1d3c26a nodeName:}" failed. No retries permitted until 2026-07-03 10:05:55.265575288 +0000 UTC m=+10.177464579 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret") pod "global-pull-secret-syncer-qqgm8" (UID: "e8ff375c-d36d-431c-ac6e-c2aef1d3c26a") : object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:51.366248 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:51.366201 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrvh7\" (UniqueName: \"kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7\") pod \"network-check-target-kxh5b\" (UID: \"76d0eec5-d0d7-4cf4-9229-b3a930c68566\") " pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:05:51.366518 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:51.366488 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 10:05:51.366518 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:51.366520 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 10:05:51.366518 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:51.366535 2573 projected.go:196] Error preparing data for projected volume kube-api-access-qrvh7 for pod openshift-network-diagnostics/network-check-target-kxh5b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:51.366834 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:51.366604 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7 podName:76d0eec5-d0d7-4cf4-9229-b3a930c68566 nodeName:}" failed. No retries permitted until 2026-07-03 10:05:55.366584307 +0000 UTC m=+10.278473593 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qrvh7" (UniqueName: "kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7") pod "network-check-target-kxh5b" (UID: "76d0eec5-d0d7-4cf4-9229-b3a930c68566") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:51.557717 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:51.557617 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:05:51.557904 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:51.557781 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:05:51.557904 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:51.557854 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:05:51.558034 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:51.557933 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:05:51.558034 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:51.557978 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:05:51.558141 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:51.558099 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:05:53.556904 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:53.556797 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:05:53.556904 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:53.556833 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:05:53.557433 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:53.556924 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:05:53.557433 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:53.556797 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:05:53.557433 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:53.557418 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:05:53.557614 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:53.557507 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:05:55.303594 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:55.302833 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs\") pod \"network-metrics-daemon-ffkpf\" (UID: \"a077a8e0-9998-4603-9c7c-f92f296c2dbf\") " pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:05:55.303594 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:55.302933 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret\") pod \"global-pull-secret-syncer-qqgm8\" (UID: \"e8ff375c-d36d-431c-ac6e-c2aef1d3c26a\") " pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:05:55.303594 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:55.303048 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:55.303594 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:55.303106 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret podName:e8ff375c-d36d-431c-ac6e-c2aef1d3c26a nodeName:}" failed. No retries permitted until 2026-07-03 10:06:03.303086453 +0000 UTC m=+18.214975753 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret") pod "global-pull-secret-syncer-qqgm8" (UID: "e8ff375c-d36d-431c-ac6e-c2aef1d3c26a") : object "kube-system"/"original-pull-secret" not registered Jul 03 10:05:55.303594 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:55.303517 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:55.303594 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:55.303565 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs podName:a077a8e0-9998-4603-9c7c-f92f296c2dbf nodeName:}" failed. No retries permitted until 2026-07-03 10:06:03.303547865 +0000 UTC m=+18.215437160 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs") pod "network-metrics-daemon-ffkpf" (UID: "a077a8e0-9998-4603-9c7c-f92f296c2dbf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:05:55.403619 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:55.403576 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrvh7\" (UniqueName: \"kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7\") pod \"network-check-target-kxh5b\" (UID: \"76d0eec5-d0d7-4cf4-9229-b3a930c68566\") " pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:05:55.403819 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:55.403752 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 10:05:55.403819 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:55.403775 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 10:05:55.403819 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:55.403787 2573 projected.go:196] Error preparing data for projected volume kube-api-access-qrvh7 for pod openshift-network-diagnostics/network-check-target-kxh5b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:55.403963 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:55.403852 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7 podName:76d0eec5-d0d7-4cf4-9229-b3a930c68566 nodeName:}" failed. No retries permitted until 2026-07-03 10:06:03.403832503 +0000 UTC m=+18.315721790 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qrvh7" (UniqueName: "kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7") pod "network-check-target-kxh5b" (UID: "76d0eec5-d0d7-4cf4-9229-b3a930c68566") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:05:55.557916 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:55.557826 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:05:55.558086 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:55.557969 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:05:55.558146 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:55.558084 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:05:55.558723 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:55.558419 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:05:55.558723 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:55.558603 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:05:55.558723 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:55.558685 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:05:57.557001 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:57.556933 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:05:57.557001 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:57.556954 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:05:57.557001 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:57.556934 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:05:57.557561 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:57.557074 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:05:57.557561 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:57.557125 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:05:57.557561 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:57.557171 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:05:59.557203 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:59.557163 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:05:59.557651 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:59.557167 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:05:59.557651 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:59.557294 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:05:59.557651 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:59.557379 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:05:59.557651 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:05:59.557173 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:05:59.557651 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:05:59.557521 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:01.557301 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:01.557252 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:01.557765 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:01.557268 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:01.557765 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:01.557390 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:01.557765 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:01.557382 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:01.557765 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:01.557510 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:01.557765 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:01.557607 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:03.361936 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:03.361892 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs\") pod \"network-metrics-daemon-ffkpf\" (UID: \"a077a8e0-9998-4603-9c7c-f92f296c2dbf\") " pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:03.362407 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:03.361990 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret\") pod \"global-pull-secret-syncer-qqgm8\" (UID: \"e8ff375c-d36d-431c-ac6e-c2aef1d3c26a\") " pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:03.362407 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:03.362063 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 10:06:03.362407 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:03.362116 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret podName:e8ff375c-d36d-431c-ac6e-c2aef1d3c26a nodeName:}" failed. No retries permitted until 2026-07-03 10:06:19.362099773 +0000 UTC m=+34.273989060 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret") pod "global-pull-secret-syncer-qqgm8" (UID: "e8ff375c-d36d-431c-ac6e-c2aef1d3c26a") : object "kube-system"/"original-pull-secret" not registered Jul 03 10:06:03.362407 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:03.362062 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:06:03.362407 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:03.362227 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs podName:a077a8e0-9998-4603-9c7c-f92f296c2dbf nodeName:}" failed. No retries permitted until 2026-07-03 10:06:19.362200961 +0000 UTC m=+34.274090251 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs") pod "network-metrics-daemon-ffkpf" (UID: "a077a8e0-9998-4603-9c7c-f92f296c2dbf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:06:03.463087 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:03.463052 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrvh7\" (UniqueName: \"kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7\") pod \"network-check-target-kxh5b\" (UID: \"76d0eec5-d0d7-4cf4-9229-b3a930c68566\") " pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:03.463271 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:03.463245 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 10:06:03.463271 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:03.463266 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 10:06:03.463392 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:03.463279 2573 projected.go:196] Error preparing data for projected volume kube-api-access-qrvh7 for pod openshift-network-diagnostics/network-check-target-kxh5b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:06:03.463392 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:03.463338 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7 podName:76d0eec5-d0d7-4cf4-9229-b3a930c68566 nodeName:}" failed. No retries permitted until 2026-07-03 10:06:19.463323806 +0000 UTC m=+34.375213093 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qrvh7" (UniqueName: "kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7") pod "network-check-target-kxh5b" (UID: "76d0eec5-d0d7-4cf4-9229-b3a930c68566") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:06:03.557071 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:03.557039 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:03.557248 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:03.557039 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:03.557248 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:03.557156 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:03.557357 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:03.557264 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:03.557357 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:03.557042 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:03.557473 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:03.557388 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:05.557377 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:05.557345 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:05.557801 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:05.557348 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:05.557801 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:05.557455 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:05.557801 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:05.557347 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:05.557801 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:05.557539 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:05.557801 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:05.557634 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:07.557184 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:07.557144 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:07.557184 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:07.557177 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:07.557696 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:07.557144 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:07.557696 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:07.557271 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:07.557696 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:07.557364 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:07.557696 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:07.557450 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:08.797281 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:08.796826 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal" event={"ID":"0928399de55edb216381d354ba304559","Type":"ContainerStarted","Data":"ab8fe4872fab0216ca4e78cac5434f66615d81aee6180eca01c2c38dc5e606d4"} Jul 03 10:06:08.798419 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:08.798395 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tjzjw" event={"ID":"0238c2d7-0223-4bbd-bc30-cb8c8901f737","Type":"ContainerStarted","Data":"024b8ad6072904caae38cf5be8201b71aada733d55087fcc19643e3655edfc20"} Jul 03 10:06:08.799915 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:08.799889 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" event={"ID":"896bd6c1-8e46-4a49-ae30-aee138404618","Type":"ContainerStarted","Data":"2234e51dc01358046897a9fd7f71acb84c551d183c51e632f0fc7ef66f336117"} Jul 03 10:06:08.803579 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:08.803550 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" event={"ID":"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0","Type":"ContainerStarted","Data":"7ec5d7b355de350654b5d7977b42e7e669b5b15b4a270dc3046c5d90ac2002a4"} Jul 03 10:06:08.803680 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:08.803611 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" event={"ID":"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0","Type":"ContainerStarted","Data":"0058ed60d7714cc94d3c9a9eabef9ed87b1c39433196157c40739aaac9676163"} Jul 03 10:06:08.803680 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:08.803649 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" event={"ID":"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0","Type":"ContainerStarted","Data":"3b48f6d45cdd982a7e17958b98e1d21b9828b8a862f66ce2bce0746601a36504"} Jul 03 10:06:08.803680 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:08.803679 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" event={"ID":"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0","Type":"ContainerStarted","Data":"90cbd7ec74ff7ce6d54cb0f68a58e82bd4345928cf9c39a67b96104739be6177"} Jul 03 10:06:08.803870 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:08.803708 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" event={"ID":"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0","Type":"ContainerStarted","Data":"709fd2a866c609c05a1f1b9ef8b4ca0e53daf46f66bfb1f002f277f6ad03d2f3"} Jul 03 10:06:08.803870 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:08.803811 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" event={"ID":"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0","Type":"ContainerStarted","Data":"701f4da49c869626b290cb2667d1985a63c515601affdcd48c8b2b09bd5abc72"} Jul 03 10:06:08.821825 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:08.821782 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-191.ec2.internal" podStartSLOduration=21.821768228 podStartE2EDuration="21.821768228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:08.821281215 +0000 UTC m=+23.733170520" watchObservedRunningTime="2026-07-03 10:06:08.821768228 +0000 UTC m=+23.733657533" Jul 03 10:06:08.844982 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:08.844945 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tr5nn" podStartSLOduration=2.700553753 podStartE2EDuration="22.844930088s" totalImagesPullingTime="20.144376335s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:08.844924716 +0000 UTC m=+23.756814021" watchObservedRunningTime="2026-07-03 10:06:08.844930088 +0000 UTC m=+23.756819392" Jul 03 10:06:09.556791 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:09.556759 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:09.556791 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:09.556794 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:09.556984 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:09.556793 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:09.556984 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:09.556854 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:09.556984 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:09.556972 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:09.557100 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:09.557026 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:09.807237 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:09.807141 2573 generic.go:356] "Generic (PLEG): container finished" podID="ce26a76a6254ac192333c41e6e1807e8" containerID="ea47b615f118d26ab7d59fbc1b7cea3b86263f20e0ef3792d9ba1f9dffed8edd" exitCode=0 Jul 03 10:06:09.807237 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:09.807216 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal" event={"ID":"ce26a76a6254ac192333c41e6e1807e8","Type":"ContainerDied","Data":"ea47b615f118d26ab7d59fbc1b7cea3b86263f20e0ef3792d9ba1f9dffed8edd"} Jul 03 10:06:09.808786 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:09.808762 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s7jq9" event={"ID":"3c7a6ee8-12fe-4c0e-a0b2-e3b828f94858","Type":"ContainerStarted","Data":"1e21deee96e8ca6c41f5eaddae8d74f08bd22978b57b57e87153c508d77e608f"} Jul 03 10:06:09.810248 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:09.810224 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" event={"ID":"c041b173-2b1a-40dd-9ef6-f9176e111c2e","Type":"ContainerStarted","Data":"624a15aff28ce53c1ae6b527e8ae07d28d88e9bb8cb72effbf6965e8284022a3"} Jul 03 10:06:09.811464 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:09.811444 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6bsh4" event={"ID":"1cda0625-6843-44e1-b61f-f8ad25fcea95","Type":"ContainerStarted","Data":"2ab2ccfd888babf0342ede5d2e8c14baacee19a470410dbb2ee6f22f1ca82de2"} Jul 03 10:06:09.812815 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:09.812792 2573 generic.go:356] "Generic (PLEG): container finished" podID="4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e" containerID="6dbe6888f8ea3eed54d728bcd5efb3eef0275f12461dbc05ee16d177aa64bd26" exitCode=0 Jul 03 10:06:09.812910 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:09.812868 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62lv9" event={"ID":"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e","Type":"ContainerDied","Data":"6dbe6888f8ea3eed54d728bcd5efb3eef0275f12461dbc05ee16d177aa64bd26"} Jul 03 10:06:09.814202 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:09.814160 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2gnl9" event={"ID":"7fcf8890-a36d-404f-95e1-631a6aa5c137","Type":"ContainerStarted","Data":"25cad1e98abc11940c279b7e3ed52bbc979ef935b5d3ff13390f5d51877f84b3"} Jul 03 10:06:09.815548 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:09.815527 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hpxfb" event={"ID":"8457043a-d27e-4962-baaa-cebc19fc88dd","Type":"ContainerStarted","Data":"c93888807b1f0c73bdef23abf88fab111b5616643eae0bb72e95ce8e6ffc80f8"} Jul 03 10:06:09.846610 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:09.845491 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tjzjw" podStartSLOduration=3.670795082 podStartE2EDuration="23.84547022s" totalImagesPullingTime="20.174675138s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:08.878932947 +0000 UTC m=+23.790822252" watchObservedRunningTime="2026-07-03 10:06:09.84547022 +0000 UTC m=+24.757359524" Jul 03 10:06:09.968418 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:09.968153 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6bsh4" podStartSLOduration=3.804176444 podStartE2EDuration="23.968131659s" totalImagesPullingTime="20.163955215s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:09.96812771 +0000 UTC m=+24.880017015" watchObservedRunningTime="2026-07-03 10:06:09.968131659 +0000 UTC m=+24.880020964" Jul 03 10:06:09.968811 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:09.968764 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-s7jq9" podStartSLOduration=11.963750817 podStartE2EDuration="23.96875248s" totalImagesPullingTime="12.005001663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:05:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:09.923060333 +0000 UTC m=+24.834949638" watchObservedRunningTime="2026-07-03 10:06:09.96875248 +0000 UTC m=+24.880641785" Jul 03 10:06:10.003223 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:10.003171 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hpxfb" podStartSLOduration=3.8664930330000002 podStartE2EDuration="24.003154398s" totalImagesPullingTime="20.136661365s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:10.002910235 +0000 UTC m=+24.914799538" watchObservedRunningTime="2026-07-03 10:06:10.003154398 +0000 UTC m=+24.915043701" Jul 03 10:06:10.026504 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:10.026453 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2gnl9" podStartSLOduration=3.898858032 podStartE2EDuration="24.026431152s" totalImagesPullingTime="20.12757312s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:10.025423305 +0000 UTC m=+24.937312612" watchObservedRunningTime="2026-07-03 10:06:10.026431152 +0000 UTC m=+24.938320457" Jul 03 10:06:10.382609 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:10.382584 2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 03 10:06:10.819716 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:10.819674 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal" event={"ID":"ce26a76a6254ac192333c41e6e1807e8","Type":"ContainerStarted","Data":"f12e229f2b009209d2a8d6e6a688450e39166279d480cd8647bf98750ba3174b"} Jul 03 10:06:10.822186 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:10.822159 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" event={"ID":"c041b173-2b1a-40dd-9ef6-f9176e111c2e","Type":"ContainerStarted","Data":"a4eb2af654613fdcc7917be48c0ead2908fa09953cc9ba0726df4c579fc53667"} Jul 03 10:06:10.840859 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:10.840817 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-191.ec2.internal" podStartSLOduration=23.840794661 podStartE2EDuration="23.840794661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:10.840440646 +0000 UTC m=+25.752329951" watchObservedRunningTime="2026-07-03 10:06:10.840794661 +0000 UTC m=+25.752683968" Jul 03 10:06:10.894632 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:10.894537 2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-03T10:06:10.382604243Z","UUID":"a14ef53c-97c2-43ef-94e8-6a0b5a1fdb72","Handler":null,"Name":"","Endpoint":""} Jul 03 10:06:10.896789 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:10.896769 2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 03 10:06:10.896902 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:10.896810 2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 03 10:06:11.556992 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:11.556954 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:11.557175 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:11.557069 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:11.557175 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:11.557076 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:11.557175 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:11.557107 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:11.557348 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:11.557202 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:11.557348 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:11.557303 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:11.602172 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:11.602145 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hpxfb_8457043a-d27e-4962-baaa-cebc19fc88dd/dns-node-resolver/0.log" Jul 03 10:06:11.830529 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:11.830500 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" event={"ID":"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0","Type":"ContainerStarted","Data":"729c1fe4c872addce5540a64291f678e61f0ec7e87cf95b912114baffa8dbb72"} Jul 03 10:06:12.360222 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:12.360195 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2gnl9_7fcf8890-a36d-404f-95e1-631a6aa5c137/node-ca/0.log" Jul 03 10:06:12.835036 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:12.834999 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" event={"ID":"c041b173-2b1a-40dd-9ef6-f9176e111c2e","Type":"ContainerStarted","Data":"efe3768af999d53e33767dd9926da6ff5e444b3732c46b9a402048ba5ac8c1ad"} Jul 03 10:06:12.859808 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:12.859753 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6vlkz" podStartSLOduration=3.940269893 podStartE2EDuration="26.859718377s" totalImagesPullingTime="22.919448484s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:12.859236564 +0000 UTC m=+27.771125870" watchObservedRunningTime="2026-07-03 10:06:12.859718377 +0000 UTC m=+27.771607683" Jul 03 10:06:13.007174 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:13.007142 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-s7jq9" Jul 03 10:06:13.007823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:13.007798 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-s7jq9" Jul 03 10:06:13.557453 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:13.557415 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:13.557453 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:13.557454 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:13.557685 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:13.557454 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:13.557685 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:13.557545 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:13.557685 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:13.557625 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:13.557848 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:13.557688 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:13.627119 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:13.627039 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-s7jq9" Jul 03 10:06:13.627539 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:13.627516 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-s7jq9" Jul 03 10:06:14.843234 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:14.842931 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" event={"ID":"ff752ee3-d640-40b6-bd9a-6532dbe9dfc0","Type":"ContainerStarted","Data":"d6bdf7559665e0bf26c34777fb7a067cd79a2a3716872cb6fa5b0bc666d53af3"} Jul 03 10:06:14.875114 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:14.875074 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" podStartSLOduration=8.652955714 podStartE2EDuration="28.875060002s" totalImagesPullingTime="20.222104288s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:14.873488623 +0000 UTC m=+29.785377927" watchObservedRunningTime="2026-07-03 10:06:14.875060002 +0000 UTC m=+29.786949306" Jul 03 10:06:15.557778 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:15.557726 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:15.558018 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:15.557726 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:15.558018 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:15.557845 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:15.558018 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:15.557728 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:15.558152 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:15.557937 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:15.558152 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:15.558031 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:15.848095 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:15.848001 2573 generic.go:356] "Generic (PLEG): container finished" podID="4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e" containerID="a108be6c2fe339e5e808f32d054f7fe3bf48f41e234f8e0a3fbefc5099928982" exitCode=0 Jul 03 10:06:15.848859 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:15.848093 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62lv9" event={"ID":"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e","Type":"ContainerDied","Data":"a108be6c2fe339e5e808f32d054f7fe3bf48f41e234f8e0a3fbefc5099928982"} Jul 03 10:06:15.848859 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:15.848672 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:06:15.848859 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:15.848759 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:06:15.848859 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:15.848806 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:06:15.865897 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:15.865872 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:06:15.866026 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:15.866009 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:06:17.557494 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:17.557459 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:17.557494 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:17.557479 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:17.558070 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:17.557467 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:17.558070 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:17.557561 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:17.558070 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:17.557626 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:17.558070 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:17.557748 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:17.854825 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:17.854716 2573 generic.go:356] "Generic (PLEG): container finished" podID="4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e" containerID="be42ed78e61a205eb795a857c5312857cddefefd119802491b3c4e11b2138feb" exitCode=0 Jul 03 10:06:17.854825 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:17.854802 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62lv9" event={"ID":"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e","Type":"ContainerDied","Data":"be42ed78e61a205eb795a857c5312857cddefefd119802491b3c4e11b2138feb"} Jul 03 10:06:19.375522 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:19.375444 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret\") pod \"global-pull-secret-syncer-qqgm8\" (UID: \"e8ff375c-d36d-431c-ac6e-c2aef1d3c26a\") " pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:19.375522 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:19.375480 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs\") pod \"network-metrics-daemon-ffkpf\" (UID: \"a077a8e0-9998-4603-9c7c-f92f296c2dbf\") " pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:19.375936 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:19.375580 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:06:19.375936 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:19.375584 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 03 10:06:19.375936 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:19.375627 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs podName:a077a8e0-9998-4603-9c7c-f92f296c2dbf nodeName:}" failed. No retries permitted until 2026-07-03 10:06:51.375613319 +0000 UTC m=+66.287502601 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs") pod "network-metrics-daemon-ffkpf" (UID: "a077a8e0-9998-4603-9c7c-f92f296c2dbf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 03 10:06:19.375936 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:19.375639 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret podName:e8ff375c-d36d-431c-ac6e-c2aef1d3c26a nodeName:}" failed. No retries permitted until 2026-07-03 10:06:51.375633401 +0000 UTC m=+66.287522683 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret") pod "global-pull-secret-syncer-qqgm8" (UID: "e8ff375c-d36d-431c-ac6e-c2aef1d3c26a") : object "kube-system"/"original-pull-secret" not registered Jul 03 10:06:19.476451 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:19.476418 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrvh7\" (UniqueName: \"kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7\") pod \"network-check-target-kxh5b\" (UID: \"76d0eec5-d0d7-4cf4-9229-b3a930c68566\") " pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:19.476609 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:19.476546 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 03 10:06:19.476609 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:19.476562 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 03 10:06:19.476609 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:19.476570 2573 projected.go:196] Error preparing data for projected volume kube-api-access-qrvh7 for pod openshift-network-diagnostics/network-check-target-kxh5b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:06:19.476698 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:19.476617 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7 podName:76d0eec5-d0d7-4cf4-9229-b3a930c68566 nodeName:}" failed. No retries permitted until 2026-07-03 10:06:51.476601562 +0000 UTC m=+66.388490863 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qrvh7" (UniqueName: "kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7") pod "network-check-target-kxh5b" (UID: "76d0eec5-d0d7-4cf4-9229-b3a930c68566") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 03 10:06:19.557255 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:19.557219 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:19.557447 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:19.557264 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:19.557447 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:19.557324 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:19.557447 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:19.557343 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:19.557447 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:19.557397 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:19.557644 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:19.557447 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:19.861829 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:19.861793 2573 generic.go:356] "Generic (PLEG): container finished" podID="4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e" containerID="df7cd6ac7e7f01a23cb375615c2c26c7ffc8f9a02da74375f3faa3163b3978b5" exitCode=0 Jul 03 10:06:19.861986 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:19.861853 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62lv9" event={"ID":"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e","Type":"ContainerDied","Data":"df7cd6ac7e7f01a23cb375615c2c26c7ffc8f9a02da74375f3faa3163b3978b5"} Jul 03 10:06:21.557127 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:21.557097 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:21.557550 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:21.557097 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:21.557550 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:21.557240 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:21.557550 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:21.557097 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:21.557550 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:21.557314 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:21.557550 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:21.557377 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:23.556908 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:23.556704 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:23.557333 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:23.556711 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:23.557333 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:23.556988 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:23.557333 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:23.557047 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:23.557333 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:23.556768 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:23.557333 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:23.557165 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:25.556884 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:25.556847 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:25.557293 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:25.556953 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:25.557293 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:25.556970 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:25.557293 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:25.556988 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:25.557293 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:25.557052 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:25.557293 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:25.557114 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:26.879263 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:26.879227 2573 generic.go:356] "Generic (PLEG): container finished" podID="4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e" containerID="8c00a3df17a80dc5cb011248d8b1f775720362f458c2d59a6f6b2750ece3b2d7" exitCode=0 Jul 03 10:06:26.879643 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:26.879270 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62lv9" event={"ID":"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e","Type":"ContainerDied","Data":"8c00a3df17a80dc5cb011248d8b1f775720362f458c2d59a6f6b2750ece3b2d7"} Jul 03 10:06:27.557580 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:27.557538 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:27.557777 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:27.557586 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:27.557777 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:27.557537 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:27.557777 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:27.557659 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:27.557777 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:27.557748 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:27.557924 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:27.557807 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:27.884938 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:27.884906 2573 generic.go:356] "Generic (PLEG): container finished" podID="4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e" containerID="38575acb874da17c97afddabddaed7c806c58e71b3875638f40cdef9aa55d441" exitCode=0 Jul 03 10:06:27.885303 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:27.884943 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62lv9" event={"ID":"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e","Type":"ContainerDied","Data":"38575acb874da17c97afddabddaed7c806c58e71b3875638f40cdef9aa55d441"} Jul 03 10:06:28.890568 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:28.890528 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-62lv9" event={"ID":"4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e","Type":"ContainerStarted","Data":"ec8965e0735b55ded3ab9b2d2f362ccccfaad0c8d3f461a39f2d856c20a7777f"} Jul 03 10:06:28.916017 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:28.915975 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-62lv9" podStartSLOduration=9.510989355 podStartE2EDuration="42.915960798s" totalImagesPullingTime="33.404971443s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:28.915514299 +0000 UTC m=+43.827403601" watchObservedRunningTime="2026-07-03 10:06:28.915960798 +0000 UTC m=+43.827850102" Jul 03 10:06:29.557338 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:29.557298 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:29.557518 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:29.557312 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:29.557518 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:29.557417 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:29.557518 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:29.557412 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:29.557518 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:29.557462 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:29.557653 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:29.557518 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:31.556884 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:31.556850 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:31.556884 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:31.556873 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:31.557362 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:31.556854 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:31.557362 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:31.556957 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:31.557362 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:31.557030 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:31.557362 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:31.557097 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:33.557281 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:33.557244 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:33.557624 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:33.557244 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:33.557624 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:33.557347 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:33.557624 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:33.557426 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:33.557624 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:33.557248 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:33.557624 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:33.557491 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:35.557231 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:35.557194 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:35.557231 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:35.557232 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:35.557634 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:35.557313 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:35.557634 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:35.557364 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:35.557634 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:35.557446 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:35.557634 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:35.557523 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:37.557528 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:37.557493 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:37.557528 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:37.557524 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:37.558005 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:37.557599 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:37.558005 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:37.557604 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:37.558005 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:37.557671 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:37.558005 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:37.557771 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:39.052273 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:39.052240 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qqgm8"] Jul 03 10:06:39.052680 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:39.052365 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:39.052680 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:39.052453 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:39.060171 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:39.060144 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ffkpf"] Jul 03 10:06:39.060305 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:39.060262 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:39.060399 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:39.060378 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:39.061130 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:39.061109 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kxh5b"] Jul 03 10:06:39.061234 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:39.061193 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:39.061299 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:39.061279 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:40.557551 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:40.557516 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:40.557935 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:40.557519 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:40.557935 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:40.557625 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:40.557935 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:40.557525 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:40.557935 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:40.557688 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:40.557935 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:40.557806 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:42.560453 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:42.560273 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:42.561002 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:42.560312 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:42.561002 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:42.560531 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kxh5b" podUID="76d0eec5-d0d7-4cf4-9229-b3a930c68566" Jul 03 10:06:42.561002 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:42.560591 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qqgm8" podUID="e8ff375c-d36d-431c-ac6e-c2aef1d3c26a" Jul 03 10:06:42.561002 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:42.560327 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:42.561002 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:06:42.560723 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffkpf" podUID="a077a8e0-9998-4603-9c7c-f92f296c2dbf" Jul 03 10:06:43.849919 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:43.849847 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-191.ec2.internal" event="NodeReady" Jul 03 10:06:43.850291 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:43.849979 2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 03 10:06:43.908514 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:43.908475 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-x27kq"] Jul 03 10:06:43.910391 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:43.910371 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bkrdf"] Jul 03 10:06:43.910523 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:43.910493 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x27kq" Jul 03 10:06:43.912137 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:43.912117 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bkrdf" Jul 03 10:06:43.913257 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:43.913235 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sgbfl\"" Jul 03 10:06:43.913364 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:43.913348 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 03 10:06:43.913428 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:43.913416 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 03 10:06:43.914826 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:43.914811 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 03 10:06:43.914956 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:43.914936 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f8d4v\"" Jul 03 10:06:43.915018 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:43.914952 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 03 10:06:43.915018 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:43.914983 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 03 10:06:43.915105 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:43.915090 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 03 10:06:43.921522 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:43.921502 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x27kq"] Jul 03 10:06:43.925528 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:43.925509 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bkrdf"] Jul 03 10:06:44.005109 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.005077 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-sbmfl"] Jul 03 10:06:44.006754 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.006725 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sbmfl" Jul 03 10:06:44.009634 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.009611 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 03 10:06:44.009634 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.009621 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 03 10:06:44.009810 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.009701 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-sl8pb\"" Jul 03 10:06:44.009810 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.009706 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 03 10:06:44.019032 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.019007 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sbmfl"] Jul 03 10:06:44.057956 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.057931 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0d094a5f-57d0-4fec-8a73-334d94229e78-config-volume\") pod \"dns-default-x27kq\" (UID: \"0d094a5f-57d0-4fec-8a73-334d94229e78\") " pod="openshift-dns/dns-default-x27kq" Jul 03 10:06:44.057956 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.057959 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ebb4964b-254e-407e-b5b1-889066572d2d-data-volume\") pod \"insights-runtime-extractor-bkrdf\" (UID: \"ebb4964b-254e-407e-b5b1-889066572d2d\") " pod="openshift-insights/insights-runtime-extractor-bkrdf" Jul 03 10:06:44.058148 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.057979 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ebb4964b-254e-407e-b5b1-889066572d2d-crio-socket\") pod \"insights-runtime-extractor-bkrdf\" (UID: \"ebb4964b-254e-407e-b5b1-889066572d2d\") " pod="openshift-insights/insights-runtime-extractor-bkrdf" Jul 03 10:06:44.058148 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.057996 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ebb4964b-254e-407e-b5b1-889066572d2d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bkrdf\" (UID: \"ebb4964b-254e-407e-b5b1-889066572d2d\") " pod="openshift-insights/insights-runtime-extractor-bkrdf" Jul 03 10:06:44.058148 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.058073 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k77kv\" (UniqueName: \"kubernetes.io/projected/0d094a5f-57d0-4fec-8a73-334d94229e78-kube-api-access-k77kv\") pod \"dns-default-x27kq\" (UID: \"0d094a5f-57d0-4fec-8a73-334d94229e78\") " pod="openshift-dns/dns-default-x27kq" Jul 03 10:06:44.058148 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.058105 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p29p\" (UniqueName: \"kubernetes.io/projected/ebb4964b-254e-407e-b5b1-889066572d2d-kube-api-access-6p29p\") pod \"insights-runtime-extractor-bkrdf\" (UID: \"ebb4964b-254e-407e-b5b1-889066572d2d\") " pod="openshift-insights/insights-runtime-extractor-bkrdf" Jul 03 10:06:44.058148 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.058133 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ebb4964b-254e-407e-b5b1-889066572d2d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bkrdf\" (UID: \"ebb4964b-254e-407e-b5b1-889066572d2d\") " pod="openshift-insights/insights-runtime-extractor-bkrdf" Jul 03 10:06:44.058295 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.058193 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d094a5f-57d0-4fec-8a73-334d94229e78-tmp-dir\") pod \"dns-default-x27kq\" (UID: \"0d094a5f-57d0-4fec-8a73-334d94229e78\") " pod="openshift-dns/dns-default-x27kq" Jul 03 10:06:44.058295 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.058221 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d094a5f-57d0-4fec-8a73-334d94229e78-metrics-tls\") pod \"dns-default-x27kq\" (UID: \"0d094a5f-57d0-4fec-8a73-334d94229e78\") " pod="openshift-dns/dns-default-x27kq" Jul 03 10:06:44.158720 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.158691 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d094a5f-57d0-4fec-8a73-334d94229e78-tmp-dir\") pod \"dns-default-x27kq\" (UID: \"0d094a5f-57d0-4fec-8a73-334d94229e78\") " pod="openshift-dns/dns-default-x27kq" Jul 03 10:06:44.158840 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.158753 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d094a5f-57d0-4fec-8a73-334d94229e78-metrics-tls\") pod \"dns-default-x27kq\" (UID: \"0d094a5f-57d0-4fec-8a73-334d94229e78\") " pod="openshift-dns/dns-default-x27kq" Jul 03 10:06:44.158840 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.158775 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4818679d-6a2e-4a1d-816b-9a52a442430a-cert\") pod \"ingress-canary-sbmfl\" (UID: \"4818679d-6a2e-4a1d-816b-9a52a442430a\") " pod="openshift-ingress-canary/ingress-canary-sbmfl" Jul 03 10:06:44.158840 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.158805 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0d094a5f-57d0-4fec-8a73-334d94229e78-config-volume\") pod \"dns-default-x27kq\" (UID: \"0d094a5f-57d0-4fec-8a73-334d94229e78\") " pod="openshift-dns/dns-default-x27kq" Jul 03 10:06:44.158840 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.158821 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ebb4964b-254e-407e-b5b1-889066572d2d-data-volume\") pod \"insights-runtime-extractor-bkrdf\" (UID: \"ebb4964b-254e-407e-b5b1-889066572d2d\") " pod="openshift-insights/insights-runtime-extractor-bkrdf" Jul 03 10:06:44.158840 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.158837 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ebb4964b-254e-407e-b5b1-889066572d2d-crio-socket\") pod \"insights-runtime-extractor-bkrdf\" (UID: \"ebb4964b-254e-407e-b5b1-889066572d2d\") " pod="openshift-insights/insights-runtime-extractor-bkrdf" Jul 03 10:06:44.159025 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.158884 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ebb4964b-254e-407e-b5b1-889066572d2d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bkrdf\" (UID: \"ebb4964b-254e-407e-b5b1-889066572d2d\") " pod="openshift-insights/insights-runtime-extractor-bkrdf" Jul 03 10:06:44.159025 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.158927 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k77kv\" (UniqueName: \"kubernetes.io/projected/0d094a5f-57d0-4fec-8a73-334d94229e78-kube-api-access-k77kv\") pod \"dns-default-x27kq\" (UID: \"0d094a5f-57d0-4fec-8a73-334d94229e78\") " pod="openshift-dns/dns-default-x27kq" Jul 03 10:06:44.159025 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.158952 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p29p\" (UniqueName: \"kubernetes.io/projected/ebb4964b-254e-407e-b5b1-889066572d2d-kube-api-access-6p29p\") pod \"insights-runtime-extractor-bkrdf\" (UID: \"ebb4964b-254e-407e-b5b1-889066572d2d\") " pod="openshift-insights/insights-runtime-extractor-bkrdf" Jul 03 10:06:44.159025 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.158986 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqnzs\" (UniqueName: \"kubernetes.io/projected/4818679d-6a2e-4a1d-816b-9a52a442430a-kube-api-access-dqnzs\") pod \"ingress-canary-sbmfl\" (UID: \"4818679d-6a2e-4a1d-816b-9a52a442430a\") " pod="openshift-ingress-canary/ingress-canary-sbmfl" Jul 03 10:06:44.159025 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.159006 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ebb4964b-254e-407e-b5b1-889066572d2d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bkrdf\" (UID: \"ebb4964b-254e-407e-b5b1-889066572d2d\") " pod="openshift-insights/insights-runtime-extractor-bkrdf" Jul 03 10:06:44.159025 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.159014 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d094a5f-57d0-4fec-8a73-334d94229e78-tmp-dir\") pod \"dns-default-x27kq\" (UID: \"0d094a5f-57d0-4fec-8a73-334d94229e78\") " pod="openshift-dns/dns-default-x27kq" Jul 03 10:06:44.159285 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.159090 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ebb4964b-254e-407e-b5b1-889066572d2d-crio-socket\") pod \"insights-runtime-extractor-bkrdf\" (UID: \"ebb4964b-254e-407e-b5b1-889066572d2d\") " pod="openshift-insights/insights-runtime-extractor-bkrdf" Jul 03 10:06:44.159285 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.159203 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ebb4964b-254e-407e-b5b1-889066572d2d-data-volume\") pod \"insights-runtime-extractor-bkrdf\" (UID: \"ebb4964b-254e-407e-b5b1-889066572d2d\") " pod="openshift-insights/insights-runtime-extractor-bkrdf" Jul 03 10:06:44.159466 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.159445 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ebb4964b-254e-407e-b5b1-889066572d2d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bkrdf\" (UID: \"ebb4964b-254e-407e-b5b1-889066572d2d\") " pod="openshift-insights/insights-runtime-extractor-bkrdf" Jul 03 10:06:44.159528 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.159484 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0d094a5f-57d0-4fec-8a73-334d94229e78-config-volume\") pod \"dns-default-x27kq\" (UID: \"0d094a5f-57d0-4fec-8a73-334d94229e78\") " pod="openshift-dns/dns-default-x27kq" Jul 03 10:06:44.162979 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.162951 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d094a5f-57d0-4fec-8a73-334d94229e78-metrics-tls\") pod \"dns-default-x27kq\" (UID: \"0d094a5f-57d0-4fec-8a73-334d94229e78\") " pod="openshift-dns/dns-default-x27kq" Jul 03 10:06:44.163081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.162994 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ebb4964b-254e-407e-b5b1-889066572d2d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bkrdf\" (UID: \"ebb4964b-254e-407e-b5b1-889066572d2d\") " pod="openshift-insights/insights-runtime-extractor-bkrdf" Jul 03 10:06:44.168518 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.168493 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p29p\" (UniqueName: \"kubernetes.io/projected/ebb4964b-254e-407e-b5b1-889066572d2d-kube-api-access-6p29p\") pod \"insights-runtime-extractor-bkrdf\" (UID: \"ebb4964b-254e-407e-b5b1-889066572d2d\") " pod="openshift-insights/insights-runtime-extractor-bkrdf" Jul 03 10:06:44.168648 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.168629 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k77kv\" (UniqueName: \"kubernetes.io/projected/0d094a5f-57d0-4fec-8a73-334d94229e78-kube-api-access-k77kv\") pod \"dns-default-x27kq\" (UID: \"0d094a5f-57d0-4fec-8a73-334d94229e78\") " pod="openshift-dns/dns-default-x27kq" Jul 03 10:06:44.221624 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.221589 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x27kq" Jul 03 10:06:44.228034 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.228014 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bkrdf" Jul 03 10:06:44.259633 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.259599 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqnzs\" (UniqueName: \"kubernetes.io/projected/4818679d-6a2e-4a1d-816b-9a52a442430a-kube-api-access-dqnzs\") pod \"ingress-canary-sbmfl\" (UID: \"4818679d-6a2e-4a1d-816b-9a52a442430a\") " pod="openshift-ingress-canary/ingress-canary-sbmfl" Jul 03 10:06:44.259827 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.259661 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4818679d-6a2e-4a1d-816b-9a52a442430a-cert\") pod \"ingress-canary-sbmfl\" (UID: \"4818679d-6a2e-4a1d-816b-9a52a442430a\") " pod="openshift-ingress-canary/ingress-canary-sbmfl" Jul 03 10:06:44.262641 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.262555 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4818679d-6a2e-4a1d-816b-9a52a442430a-cert\") pod \"ingress-canary-sbmfl\" (UID: \"4818679d-6a2e-4a1d-816b-9a52a442430a\") " pod="openshift-ingress-canary/ingress-canary-sbmfl" Jul 03 10:06:44.269298 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.269272 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqnzs\" (UniqueName: \"kubernetes.io/projected/4818679d-6a2e-4a1d-816b-9a52a442430a-kube-api-access-dqnzs\") pod \"ingress-canary-sbmfl\" (UID: \"4818679d-6a2e-4a1d-816b-9a52a442430a\") " pod="openshift-ingress-canary/ingress-canary-sbmfl" Jul 03 10:06:44.315863 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.315551 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sbmfl" Jul 03 10:06:44.365817 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.365788 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bkrdf"] Jul 03 10:06:44.371214 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:06:44.371155 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebb4964b_254e_407e_b5b1_889066572d2d.slice/crio-68d6c6b976c162aac1d687817f2050a58564c49bca1d036b6c7378d6fc15c770 WatchSource:0}: Error finding container 68d6c6b976c162aac1d687817f2050a58564c49bca1d036b6c7378d6fc15c770: Status 404 returned error can't find the container with id 68d6c6b976c162aac1d687817f2050a58564c49bca1d036b6c7378d6fc15c770 Jul 03 10:06:44.383300 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.383276 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x27kq"] Jul 03 10:06:44.385117 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:06:44.385092 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d094a5f_57d0_4fec_8a73_334d94229e78.slice/crio-da80f3ee1c7be7c9831d53f0ef120d04b095e34923338d9724e4b11bdeb64fd9 WatchSource:0}: Error finding container da80f3ee1c7be7c9831d53f0ef120d04b095e34923338d9724e4b11bdeb64fd9: Status 404 returned error can't find the container with id da80f3ee1c7be7c9831d53f0ef120d04b095e34923338d9724e4b11bdeb64fd9 Jul 03 10:06:44.450240 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.450172 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sbmfl"] Jul 03 10:06:44.453190 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:06:44.453166 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4818679d_6a2e_4a1d_816b_9a52a442430a.slice/crio-e60cdf16082e2eb2e3af925589cf04da80ccdb2e3237c0c3a1d7fb01ce931c0c WatchSource:0}: Error finding container e60cdf16082e2eb2e3af925589cf04da80ccdb2e3237c0c3a1d7fb01ce931c0c: Status 404 returned error can't find the container with id e60cdf16082e2eb2e3af925589cf04da80ccdb2e3237c0c3a1d7fb01ce931c0c Jul 03 10:06:44.557574 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.557536 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:44.557775 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.557534 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:44.557775 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.557536 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:44.561180 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.561155 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 03 10:06:44.561180 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.561158 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 03 10:06:44.561385 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.561215 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 03 10:06:44.561385 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.561231 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xh5d7\"" Jul 03 10:06:44.561385 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.561281 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 03 10:06:44.561385 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.561228 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-njhrs\"" Jul 03 10:06:44.930766 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.930715 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sbmfl" event={"ID":"4818679d-6a2e-4a1d-816b-9a52a442430a","Type":"ContainerStarted","Data":"e60cdf16082e2eb2e3af925589cf04da80ccdb2e3237c0c3a1d7fb01ce931c0c"} Jul 03 10:06:44.934636 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.934603 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bkrdf" event={"ID":"ebb4964b-254e-407e-b5b1-889066572d2d","Type":"ContainerStarted","Data":"68d6c6b976c162aac1d687817f2050a58564c49bca1d036b6c7378d6fc15c770"} Jul 03 10:06:44.936609 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:44.936525 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x27kq" event={"ID":"0d094a5f-57d0-4fec-8a73-334d94229e78","Type":"ContainerStarted","Data":"da80f3ee1c7be7c9831d53f0ef120d04b095e34923338d9724e4b11bdeb64fd9"} Jul 03 10:06:46.968275 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:46.968246 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-857f869b78-dpzgg"] Jul 03 10:06:46.976414 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:46.976021 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:46.979262 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:46.979240 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 03 10:06:46.979403 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:46.979385 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 03 10:06:46.979946 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:46.979922 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 03 10:06:46.980070 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:46.979978 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-b5rfb\"" Jul 03 10:06:46.980132 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:46.979979 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 03 10:06:46.980132 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:46.979922 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 03 10:06:46.980132 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:46.980081 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 03 10:06:46.980657 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:46.980638 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 03 10:06:46.986266 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:46.986241 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-857f869b78-dpzgg"] Jul 03 10:06:46.986950 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:46.986927 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 03 10:06:47.086119 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.086080 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2g22\" (UniqueName: \"kubernetes.io/projected/86c2c605-065a-454b-bcb2-5cc057993daf-kube-api-access-q2g22\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.086119 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.086124 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-oauth-serving-cert\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.086343 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.086167 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-trusted-ca-bundle\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.086343 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.086285 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86c2c605-065a-454b-bcb2-5cc057993daf-console-oauth-config\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.086437 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.086379 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86c2c605-065a-454b-bcb2-5cc057993daf-console-serving-cert\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.086437 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.086405 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-console-config\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.086437 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.086429 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-service-ca\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.187040 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.187001 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86c2c605-065a-454b-bcb2-5cc057993daf-console-serving-cert\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.187040 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.187043 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-console-config\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.187278 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.187219 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-service-ca\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.187335 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.187281 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2g22\" (UniqueName: \"kubernetes.io/projected/86c2c605-065a-454b-bcb2-5cc057993daf-kube-api-access-q2g22\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.187335 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.187319 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-oauth-serving-cert\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.187417 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.187370 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-trusted-ca-bundle\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.187464 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.187418 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86c2c605-065a-454b-bcb2-5cc057993daf-console-oauth-config\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.187936 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.187802 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-console-config\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.187936 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.187890 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-service-ca\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.188307 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.188282 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-oauth-serving-cert\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.188418 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.188344 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-trusted-ca-bundle\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.190152 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.190105 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86c2c605-065a-454b-bcb2-5cc057993daf-console-serving-cert\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.190340 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.190319 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86c2c605-065a-454b-bcb2-5cc057993daf-console-oauth-config\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.208940 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.208914 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2g22\" (UniqueName: \"kubernetes.io/projected/86c2c605-065a-454b-bcb2-5cc057993daf-kube-api-access-q2g22\") pod \"console-857f869b78-dpzgg\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.289942 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.289852 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:47.868411 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:47.868385 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-phvfm" Jul 03 10:06:48.024265 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.024230 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n"] Jul 03 10:06:48.044468 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.044440 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n" Jul 03 10:06:48.046451 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.046426 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n"] Jul 03 10:06:48.048064 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.048043 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 03 10:06:48.051456 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.051428 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gfjrs"] Jul 03 10:06:48.052082 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.052049 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 03 10:06:48.055168 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.055084 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 03 10:06:48.055318 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.055299 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 03 10:06:48.055380 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.055323 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 03 10:06:48.057189 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.056926 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-5k2vr\"" Jul 03 10:06:48.070879 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.070849 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4"] Jul 03 10:06:48.071030 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.070993 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.079563 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.079294 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xbqqd\"" Jul 03 10:06:48.079641 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.079620 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 03 10:06:48.079858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.079838 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 03 10:06:48.080064 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.080038 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 03 10:06:48.081175 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.081157 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4"] Jul 03 10:06:48.081270 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.081260 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" Jul 03 10:06:48.085073 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.085042 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 03 10:06:48.085194 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.085107 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 03 10:06:48.085194 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.085128 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-4p5gj\"" Jul 03 10:06:48.085194 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.085131 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 03 10:06:48.194397 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.194357 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-sys\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.194397 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.194399 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b2ebc75-c820-4e81-9704-e2580f24da82-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-vr6b4\" (UID: \"7b2ebc75-c820-4e81-9704-e2580f24da82\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" Jul 03 10:06:48.194654 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.194419 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-node-exporter-textfile\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.194654 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.194505 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b2ebc75-c820-4e81-9704-e2580f24da82-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-vr6b4\" (UID: \"7b2ebc75-c820-4e81-9704-e2580f24da82\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" Jul 03 10:06:48.194654 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.194553 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f73dc61-257e-4a34-80ef-8c86c5d4511e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-7gn6n\" (UID: \"8f73dc61-257e-4a34-80ef-8c86c5d4511e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n" Jul 03 10:06:48.194654 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.194572 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs65j\" (UniqueName: \"kubernetes.io/projected/7b2ebc75-c820-4e81-9704-e2580f24da82-kube-api-access-fs65j\") pod \"kube-state-metrics-7c9dbb8b7d-vr6b4\" (UID: \"7b2ebc75-c820-4e81-9704-e2580f24da82\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" Jul 03 10:06:48.194654 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.194611 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-node-exporter-wtmp\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.194654 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.194643 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-node-exporter-tls\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.194951 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.194675 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.194951 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.194721 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7b2ebc75-c820-4e81-9704-e2580f24da82-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-vr6b4\" (UID: \"7b2ebc75-c820-4e81-9704-e2580f24da82\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" Jul 03 10:06:48.194951 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.194803 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sknxc\" (UniqueName: \"kubernetes.io/projected/8f73dc61-257e-4a34-80ef-8c86c5d4511e-kube-api-access-sknxc\") pod \"openshift-state-metrics-54cf96dfcf-7gn6n\" (UID: \"8f73dc61-257e-4a34-80ef-8c86c5d4511e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n" Jul 03 10:06:48.194951 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.194827 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-metrics-client-ca\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.194951 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.194864 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7b2ebc75-c820-4e81-9704-e2580f24da82-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-vr6b4\" (UID: \"7b2ebc75-c820-4e81-9704-e2580f24da82\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" Jul 03 10:06:48.194951 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.194892 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-node-exporter-accelerators-collector-config\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.194951 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.194927 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8f73dc61-257e-4a34-80ef-8c86c5d4511e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-7gn6n\" (UID: \"8f73dc61-257e-4a34-80ef-8c86c5d4511e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n" Jul 03 10:06:48.194951 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.194956 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f73dc61-257e-4a34-80ef-8c86c5d4511e-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-7gn6n\" (UID: \"8f73dc61-257e-4a34-80ef-8c86c5d4511e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n" Jul 03 10:06:48.195253 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.194974 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdjz2\" (UniqueName: \"kubernetes.io/projected/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-kube-api-access-xdjz2\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.195253 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.195007 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-root\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.195253 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.195037 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7b2ebc75-c820-4e81-9704-e2580f24da82-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-vr6b4\" (UID: \"7b2ebc75-c820-4e81-9704-e2580f24da82\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" Jul 03 10:06:48.295702 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.295658 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-root\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.295702 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.295703 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7b2ebc75-c820-4e81-9704-e2580f24da82-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-vr6b4\" (UID: \"7b2ebc75-c820-4e81-9704-e2580f24da82\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" Jul 03 10:06:48.296010 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.295728 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-sys\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.296010 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.295819 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-root\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.296010 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.295873 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b2ebc75-c820-4e81-9704-e2580f24da82-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-vr6b4\" (UID: \"7b2ebc75-c820-4e81-9704-e2580f24da82\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" Jul 03 10:06:48.296010 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.295903 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-node-exporter-textfile\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.296010 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.295935 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b2ebc75-c820-4e81-9704-e2580f24da82-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-vr6b4\" (UID: \"7b2ebc75-c820-4e81-9704-e2580f24da82\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" Jul 03 10:06:48.296010 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.295975 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f73dc61-257e-4a34-80ef-8c86c5d4511e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-7gn6n\" (UID: \"8f73dc61-257e-4a34-80ef-8c86c5d4511e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n" Jul 03 10:06:48.296010 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.296003 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs65j\" (UniqueName: \"kubernetes.io/projected/7b2ebc75-c820-4e81-9704-e2580f24da82-kube-api-access-fs65j\") pod \"kube-state-metrics-7c9dbb8b7d-vr6b4\" (UID: \"7b2ebc75-c820-4e81-9704-e2580f24da82\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" Jul 03 10:06:48.296349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.296049 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-node-exporter-wtmp\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.296349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.296096 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-node-exporter-tls\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.296349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.296135 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.296349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.296168 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7b2ebc75-c820-4e81-9704-e2580f24da82-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-vr6b4\" (UID: \"7b2ebc75-c820-4e81-9704-e2580f24da82\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" Jul 03 10:06:48.296349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.296196 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sknxc\" (UniqueName: \"kubernetes.io/projected/8f73dc61-257e-4a34-80ef-8c86c5d4511e-kube-api-access-sknxc\") pod \"openshift-state-metrics-54cf96dfcf-7gn6n\" (UID: \"8f73dc61-257e-4a34-80ef-8c86c5d4511e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n" Jul 03 10:06:48.296349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.296224 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-metrics-client-ca\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.296349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.296228 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-node-exporter-textfile\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.296349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.296275 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7b2ebc75-c820-4e81-9704-e2580f24da82-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-vr6b4\" (UID: \"7b2ebc75-c820-4e81-9704-e2580f24da82\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" Jul 03 10:06:48.296349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.296305 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-node-exporter-accelerators-collector-config\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.296349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.296338 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8f73dc61-257e-4a34-80ef-8c86c5d4511e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-7gn6n\" (UID: \"8f73dc61-257e-4a34-80ef-8c86c5d4511e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n" Jul 03 10:06:48.296349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.296343 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-node-exporter-wtmp\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.297943 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.296373 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f73dc61-257e-4a34-80ef-8c86c5d4511e-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-7gn6n\" (UID: \"8f73dc61-257e-4a34-80ef-8c86c5d4511e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n" Jul 03 10:06:48.297943 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.296405 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdjz2\" (UniqueName: \"kubernetes.io/projected/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-kube-api-access-xdjz2\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.297943 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.296563 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7b2ebc75-c820-4e81-9704-e2580f24da82-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-vr6b4\" (UID: \"7b2ebc75-c820-4e81-9704-e2580f24da82\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" Jul 03 10:06:48.297943 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.296694 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b2ebc75-c820-4e81-9704-e2580f24da82-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-vr6b4\" (UID: \"7b2ebc75-c820-4e81-9704-e2580f24da82\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" Jul 03 10:06:48.297943 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.296811 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-sys\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.297943 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.297481 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7b2ebc75-c820-4e81-9704-e2580f24da82-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-vr6b4\" (UID: \"7b2ebc75-c820-4e81-9704-e2580f24da82\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" Jul 03 10:06:48.297943 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.297548 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-metrics-client-ca\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.297943 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.297682 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8f73dc61-257e-4a34-80ef-8c86c5d4511e-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-7gn6n\" (UID: \"8f73dc61-257e-4a34-80ef-8c86c5d4511e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n" Jul 03 10:06:48.297943 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.297936 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-node-exporter-accelerators-collector-config\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.299236 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.299204 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b2ebc75-c820-4e81-9704-e2580f24da82-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-vr6b4\" (UID: \"7b2ebc75-c820-4e81-9704-e2580f24da82\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" Jul 03 10:06:48.299354 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.299213 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8f73dc61-257e-4a34-80ef-8c86c5d4511e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-7gn6n\" (UID: \"8f73dc61-257e-4a34-80ef-8c86c5d4511e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n" Jul 03 10:06:48.299881 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.299847 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.299980 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.299908 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-node-exporter-tls\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.300178 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.300148 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8f73dc61-257e-4a34-80ef-8c86c5d4511e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-7gn6n\" (UID: \"8f73dc61-257e-4a34-80ef-8c86c5d4511e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n" Jul 03 10:06:48.300503 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.300477 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7b2ebc75-c820-4e81-9704-e2580f24da82-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-vr6b4\" (UID: \"7b2ebc75-c820-4e81-9704-e2580f24da82\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" Jul 03 10:06:48.319213 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.319179 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdjz2\" (UniqueName: \"kubernetes.io/projected/cdeaefc1-cb0f-4c41-8e6d-14918dfe9318-kube-api-access-xdjz2\") pod \"node-exporter-gfjrs\" (UID: \"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318\") " pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.323960 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.323932 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sknxc\" (UniqueName: \"kubernetes.io/projected/8f73dc61-257e-4a34-80ef-8c86c5d4511e-kube-api-access-sknxc\") pod \"openshift-state-metrics-54cf96dfcf-7gn6n\" (UID: \"8f73dc61-257e-4a34-80ef-8c86c5d4511e\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n" Jul 03 10:06:48.337167 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.337140 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs65j\" (UniqueName: \"kubernetes.io/projected/7b2ebc75-c820-4e81-9704-e2580f24da82-kube-api-access-fs65j\") pod \"kube-state-metrics-7c9dbb8b7d-vr6b4\" (UID: \"7b2ebc75-c820-4e81-9704-e2580f24da82\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" Jul 03 10:06:48.358903 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.358868 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n" Jul 03 10:06:48.385040 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.384973 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gfjrs" Jul 03 10:06:48.392282 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:48.392258 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" Jul 03 10:06:49.000635 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.000561 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 03 10:06:49.054474 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.054441 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 03 10:06:49.054918 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.054570 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.057645 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.057622 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-hp98j\"" Jul 03 10:06:49.057802 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.057778 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 03 10:06:49.057884 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.057801 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 03 10:06:49.057884 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.057872 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 03 10:06:49.057988 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.057905 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 03 10:06:49.058250 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.058209 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 03 10:06:49.058376 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.058302 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 03 10:06:49.058441 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.058418 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 03 10:06:49.058554 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.058525 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 03 10:06:49.061166 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.061149 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 03 10:06:49.209113 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.205586 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.209113 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.205677 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8c5f\" (UniqueName: \"kubernetes.io/projected/4615fd38-b169-4b84-90f0-ca1ab940acd5-kube-api-access-p8c5f\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.209113 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.205716 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-config-volume\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.209113 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.205764 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4615fd38-b169-4b84-90f0-ca1ab940acd5-config-out\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.209113 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.205840 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.209113 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.205896 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4615fd38-b169-4b84-90f0-ca1ab940acd5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.209113 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.205965 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-web-config\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.209113 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.206006 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4615fd38-b169-4b84-90f0-ca1ab940acd5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.209113 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.206045 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.209113 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.206097 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.209113 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.206144 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4615fd38-b169-4b84-90f0-ca1ab940acd5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.209113 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.206191 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.209113 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.206229 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4615fd38-b169-4b84-90f0-ca1ab940acd5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.307026 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.306940 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.307026 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.306999 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4615fd38-b169-4b84-90f0-ca1ab940acd5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.307256 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.307044 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.307256 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.307076 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4615fd38-b169-4b84-90f0-ca1ab940acd5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.307256 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.307158 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.307256 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.307192 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8c5f\" (UniqueName: \"kubernetes.io/projected/4615fd38-b169-4b84-90f0-ca1ab940acd5-kube-api-access-p8c5f\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.307256 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.307219 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-config-volume\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.307256 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.307244 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4615fd38-b169-4b84-90f0-ca1ab940acd5-config-out\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.307539 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.307270 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.307539 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.307294 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4615fd38-b169-4b84-90f0-ca1ab940acd5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.307539 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.307323 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-web-config\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.307539 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.307350 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4615fd38-b169-4b84-90f0-ca1ab940acd5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.307539 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.307396 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.308654 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.308318 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4615fd38-b169-4b84-90f0-ca1ab940acd5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.308782 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.308674 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4615fd38-b169-4b84-90f0-ca1ab940acd5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.308782 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.308705 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4615fd38-b169-4b84-90f0-ca1ab940acd5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.310385 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.310355 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.310601 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.310553 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.311304 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.311265 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-web-config\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.311704 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.311677 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4615fd38-b169-4b84-90f0-ca1ab940acd5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.312004 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.311981 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.312246 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.312212 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-config-volume\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.312442 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.312245 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.312638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.312615 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.313491 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.313463 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4615fd38-b169-4b84-90f0-ca1ab940acd5-config-out\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.329018 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.328988 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8c5f\" (UniqueName: \"kubernetes.io/projected/4615fd38-b169-4b84-90f0-ca1ab940acd5-kube-api-access-p8c5f\") pod \"alertmanager-main-0\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.366349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.366310 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:06:49.751487 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:06:49.751401 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdeaefc1_cb0f_4c41_8e6d_14918dfe9318.slice/crio-ec29149a70cab2c2196e2483ab8fd4fdf36801ce1921977360766587b3f9e393 WatchSource:0}: Error finding container ec29149a70cab2c2196e2483ab8fd4fdf36801ce1921977360766587b3f9e393: Status 404 returned error can't find the container with id ec29149a70cab2c2196e2483ab8fd4fdf36801ce1921977360766587b3f9e393 Jul 03 10:06:49.928947 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.928609 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-857f869b78-dpzgg"] Jul 03 10:06:49.932344 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:06:49.932305 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86c2c605_065a_454b_bcb2_5cc057993daf.slice/crio-55bb102bbc60b47004563299026ab6c511efbff7155da6638c086da21ed1cf6e WatchSource:0}: Error finding container 55bb102bbc60b47004563299026ab6c511efbff7155da6638c086da21ed1cf6e: Status 404 returned error can't find the container with id 55bb102bbc60b47004563299026ab6c511efbff7155da6638c086da21ed1cf6e Jul 03 10:06:49.953306 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.951680 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n"] Jul 03 10:06:49.957262 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.953577 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4"] Jul 03 10:06:49.957262 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:06:49.957223 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b2ebc75_c820_4e81_9704_e2580f24da82.slice/crio-d42a9e83955102f3dff5bfe5ca6b56964bed7f8aaec50040549294b9945a7256 WatchSource:0}: Error finding container d42a9e83955102f3dff5bfe5ca6b56964bed7f8aaec50040549294b9945a7256: Status 404 returned error can't find the container with id d42a9e83955102f3dff5bfe5ca6b56964bed7f8aaec50040549294b9945a7256 Jul 03 10:06:49.966993 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.966722 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sbmfl" event={"ID":"4818679d-6a2e-4a1d-816b-9a52a442430a","Type":"ContainerStarted","Data":"e0e16942f71fbcef88995521da7d2a8fab3d0a45168191ed378263701df0077a"} Jul 03 10:06:49.971478 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.971301 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gfjrs" event={"ID":"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318","Type":"ContainerStarted","Data":"ec29149a70cab2c2196e2483ab8fd4fdf36801ce1921977360766587b3f9e393"} Jul 03 10:06:49.981171 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.981139 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857f869b78-dpzgg" event={"ID":"86c2c605-065a-454b-bcb2-5cc057993daf","Type":"ContainerStarted","Data":"55bb102bbc60b47004563299026ab6c511efbff7155da6638c086da21ed1cf6e"} Jul 03 10:06:49.985184 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.984327 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x27kq" event={"ID":"0d094a5f-57d0-4fec-8a73-334d94229e78","Type":"ContainerStarted","Data":"71e3244ac5d59a8d4f6ead5191646b7892bce5dec8db103445994a796f1e05db"} Jul 03 10:06:49.990416 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.988764 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bkrdf" event={"ID":"ebb4964b-254e-407e-b5b1-889066572d2d","Type":"ContainerStarted","Data":"8ae4904876e67967527ce49c06ce7a19e1d85c2514b2e5a61206e38f5ed9a56b"} Jul 03 10:06:49.990416 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.990146 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-sbmfl" podStartSLOduration=1.695099028 podStartE2EDuration="6.990127129s" totalImagesPullingTime="5.295028101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:06:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:49.987260598 +0000 UTC m=+64.899149903" watchObservedRunningTime="2026-07-03 10:06:49.990127129 +0000 UTC m=+64.902016435" Jul 03 10:06:49.994807 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:49.994453 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 03 10:06:49.999219 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:06:49.999189 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4615fd38_b169_4b84_90f0_ca1ab940acd5.slice/crio-db4216ee64485141d352f2b3e383eea2dda217a9bca0a87e5e18694a9e8bbd8b WatchSource:0}: Error finding container db4216ee64485141d352f2b3e383eea2dda217a9bca0a87e5e18694a9e8bbd8b: Status 404 returned error can't find the container with id db4216ee64485141d352f2b3e383eea2dda217a9bca0a87e5e18694a9e8bbd8b Jul 03 10:06:51.008171 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.006507 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7f8d657f7c-j76pt"] Jul 03 10:06:51.010682 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.009449 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.010682 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.010611 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x27kq" event={"ID":"0d094a5f-57d0-4fec-8a73-334d94229e78","Type":"ContainerStarted","Data":"27ebf6a93f1bf1508ddbf9ef588015eda1aed5d62568837a3df70c9a24198f4c"} Jul 03 10:06:51.011029 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.011008 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-x27kq" Jul 03 10:06:51.013061 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.013006 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 03 10:06:51.013823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.013240 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 03 10:06:51.013823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.013387 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-92es3jh0ouvuo\"" Jul 03 10:06:51.013823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.013501 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 03 10:06:51.014523 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.014126 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 03 10:06:51.014523 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.014261 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-dxlq2\"" Jul 03 10:06:51.014523 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.014359 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 03 10:06:51.015884 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.015856 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n" event={"ID":"8f73dc61-257e-4a34-80ef-8c86c5d4511e","Type":"ContainerStarted","Data":"e52494d165655ab347f0a197c8502000a8aea9d58ed1059675cb85f6fe1a964d"} Jul 03 10:06:51.015986 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.015917 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n" event={"ID":"8f73dc61-257e-4a34-80ef-8c86c5d4511e","Type":"ContainerStarted","Data":"b27838f12552b4d5088127a9f2bffee54336d7924eaf2b1fb87310ddd2296f88"} Jul 03 10:06:51.015986 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.015950 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n" event={"ID":"8f73dc61-257e-4a34-80ef-8c86c5d4511e","Type":"ContainerStarted","Data":"c732de9d4231fa304ca0c284a95f882573a67f94bfd34b0b313b19067f9ddb8a"} Jul 03 10:06:51.018277 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.018255 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" event={"ID":"7b2ebc75-c820-4e81-9704-e2580f24da82","Type":"ContainerStarted","Data":"d42a9e83955102f3dff5bfe5ca6b56964bed7f8aaec50040549294b9945a7256"} Jul 03 10:06:51.024116 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.024056 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bkrdf" event={"ID":"ebb4964b-254e-407e-b5b1-889066572d2d","Type":"ContainerStarted","Data":"3e1af46e71e6d9378a03139c9af6df2acfe3486732e97764816430fb947366e3"} Jul 03 10:06:51.024242 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.024227 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7f8d657f7c-j76pt"] Jul 03 10:06:51.026869 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.026807 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4615fd38-b169-4b84-90f0-ca1ab940acd5","Type":"ContainerStarted","Data":"db4216ee64485141d352f2b3e383eea2dda217a9bca0a87e5e18694a9e8bbd8b"} Jul 03 10:06:51.124225 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.122459 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92357b58-c179-4495-be4f-295b61cc0ad5-metrics-client-ca\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.124225 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.122511 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/92357b58-c179-4495-be4f-295b61cc0ad5-secret-grpc-tls\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.124225 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.122606 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/92357b58-c179-4495-be4f-295b61cc0ad5-secret-thanos-querier-tls\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.124225 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.122631 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92357b58-c179-4495-be4f-295b61cc0ad5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.124225 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.122665 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92357b58-c179-4495-be4f-295b61cc0ad5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.124225 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.122794 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgzmd\" (UniqueName: \"kubernetes.io/projected/92357b58-c179-4495-be4f-295b61cc0ad5-kube-api-access-dgzmd\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.124225 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.122835 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/92357b58-c179-4495-be4f-295b61cc0ad5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.124225 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.122903 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/92357b58-c179-4495-be4f-295b61cc0ad5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.224107 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.224024 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/92357b58-c179-4495-be4f-295b61cc0ad5-secret-thanos-querier-tls\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.224107 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.224076 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92357b58-c179-4495-be4f-295b61cc0ad5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.224333 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.224108 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92357b58-c179-4495-be4f-295b61cc0ad5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.224333 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.224170 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgzmd\" (UniqueName: \"kubernetes.io/projected/92357b58-c179-4495-be4f-295b61cc0ad5-kube-api-access-dgzmd\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.224333 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.224211 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/92357b58-c179-4495-be4f-295b61cc0ad5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.224333 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.224249 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/92357b58-c179-4495-be4f-295b61cc0ad5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.224333 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.224275 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92357b58-c179-4495-be4f-295b61cc0ad5-metrics-client-ca\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.224333 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.224302 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/92357b58-c179-4495-be4f-295b61cc0ad5-secret-grpc-tls\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.225704 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.225654 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92357b58-c179-4495-be4f-295b61cc0ad5-metrics-client-ca\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.227711 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.227662 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/92357b58-c179-4495-be4f-295b61cc0ad5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.228158 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.228082 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/92357b58-c179-4495-be4f-295b61cc0ad5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.228913 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.228871 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/92357b58-c179-4495-be4f-295b61cc0ad5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.231034 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.230992 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/92357b58-c179-4495-be4f-295b61cc0ad5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.231403 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.231378 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/92357b58-c179-4495-be4f-295b61cc0ad5-secret-grpc-tls\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.232315 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.232291 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/92357b58-c179-4495-be4f-295b61cc0ad5-secret-thanos-querier-tls\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.234708 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.234686 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgzmd\" (UniqueName: \"kubernetes.io/projected/92357b58-c179-4495-be4f-295b61cc0ad5-kube-api-access-dgzmd\") pod \"thanos-querier-7f8d657f7c-j76pt\" (UID: \"92357b58-c179-4495-be4f-295b61cc0ad5\") " pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.325239 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.324938 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:06:51.425795 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.425754 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret\") pod \"global-pull-secret-syncer-qqgm8\" (UID: \"e8ff375c-d36d-431c-ac6e-c2aef1d3c26a\") " pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:51.425795 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.425807 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs\") pod \"network-metrics-daemon-ffkpf\" (UID: \"a077a8e0-9998-4603-9c7c-f92f296c2dbf\") " pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:51.429289 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.429264 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 03 10:06:51.429446 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.429323 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 03 10:06:51.439446 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.439390 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a077a8e0-9998-4603-9c7c-f92f296c2dbf-metrics-certs\") pod \"network-metrics-daemon-ffkpf\" (UID: \"a077a8e0-9998-4603-9c7c-f92f296c2dbf\") " pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:51.439827 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.439801 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8ff375c-d36d-431c-ac6e-c2aef1d3c26a-original-pull-secret\") pod \"global-pull-secret-syncer-qqgm8\" (UID: \"e8ff375c-d36d-431c-ac6e-c2aef1d3c26a\") " pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:51.468644 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.468613 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qqgm8" Jul 03 10:06:51.481228 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.481161 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xh5d7\"" Jul 03 10:06:51.487963 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.487928 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffkpf" Jul 03 10:06:51.526596 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.526517 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrvh7\" (UniqueName: \"kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7\") pod \"network-check-target-kxh5b\" (UID: \"76d0eec5-d0d7-4cf4-9229-b3a930c68566\") " pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:51.530066 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.529851 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 03 10:06:51.540727 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.540498 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 03 10:06:51.551838 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.551761 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrvh7\" (UniqueName: \"kubernetes.io/projected/76d0eec5-d0d7-4cf4-9229-b3a930c68566-kube-api-access-qrvh7\") pod \"network-check-target-kxh5b\" (UID: \"76d0eec5-d0d7-4cf4-9229-b3a930c68566\") " pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:51.776388 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.776129 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-njhrs\"" Jul 03 10:06:51.783697 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:51.783662 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:06:52.033759 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.033661 2573 generic.go:356] "Generic (PLEG): container finished" podID="cdeaefc1-cb0f-4c41-8e6d-14918dfe9318" containerID="686e1ed1892335745ca35c0584da16903e2fe52f12d640927f86db9921be6b03" exitCode=0 Jul 03 10:06:52.034141 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.033760 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gfjrs" event={"ID":"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318","Type":"ContainerDied","Data":"686e1ed1892335745ca35c0584da16903e2fe52f12d640927f86db9921be6b03"} Jul 03 10:06:52.061020 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.060968 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-x27kq" podStartSLOduration=3.7157022360000003 podStartE2EDuration="9.060949216s" totalImagesPullingTime="5.34524698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:06:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:51.079922705 +0000 UTC m=+65.991812002" watchObservedRunningTime="2026-07-03 10:06:52.060949216 +0000 UTC m=+66.972838500" Jul 03 10:06:52.666003 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.665966 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-865d4c69d7-pppdd"] Jul 03 10:06:52.668306 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.668280 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.674968 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.674923 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 03 10:06:52.675102 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.675063 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-bgaklg8tgs4g4\"" Jul 03 10:06:52.675215 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.675199 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 03 10:06:52.675346 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.675327 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 03 10:06:52.675533 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.675503 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 03 10:06:52.675931 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.675719 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-zz2cj\"" Jul 03 10:06:52.697461 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.697425 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-865d4c69d7-pppdd"] Jul 03 10:06:52.718278 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.718249 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-ncl65"] Jul 03 10:06:52.720782 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.720762 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ncl65" Jul 03 10:06:52.723633 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.723608 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 03 10:06:52.724196 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.724178 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-zh56m\"" Jul 03 10:06:52.737507 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.736679 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-ncl65"] Jul 03 10:06:52.838275 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.838244 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/67615ad3-a3cc-45e9-a148-c0476d653e11-secret-metrics-server-client-certs\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.838439 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.838295 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/67615ad3-a3cc-45e9-a148-c0476d653e11-secret-metrics-server-tls\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.838439 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.838328 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67615ad3-a3cc-45e9-a148-c0476d653e11-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.838439 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.838385 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/67615ad3-a3cc-45e9-a148-c0476d653e11-client-ca-bundle\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.838587 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.838479 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eeccf333-5485-44f1-b0c6-617be538c81b-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-ncl65\" (UID: \"eeccf333-5485-44f1-b0c6-617be538c81b\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ncl65" Jul 03 10:06:52.838587 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.838547 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbxh6\" (UniqueName: \"kubernetes.io/projected/eeccf333-5485-44f1-b0c6-617be538c81b-kube-api-access-xbxh6\") pod \"monitoring-plugin-d5dcff7cf-ncl65\" (UID: \"eeccf333-5485-44f1-b0c6-617be538c81b\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ncl65" Jul 03 10:06:52.838674 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.838592 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/67615ad3-a3cc-45e9-a148-c0476d653e11-metrics-server-audit-profiles\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.838674 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.838619 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6579\" (UniqueName: \"kubernetes.io/projected/67615ad3-a3cc-45e9-a148-c0476d653e11-kube-api-access-s6579\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.838674 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.838670 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/67615ad3-a3cc-45e9-a148-c0476d653e11-audit-log\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.891697 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.891664 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77849d98d-pxzz4"] Jul 03 10:06:52.896126 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.896105 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:52.918009 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.917942 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77849d98d-pxzz4"] Jul 03 10:06:52.939562 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.939524 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eeccf333-5485-44f1-b0c6-617be538c81b-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-ncl65\" (UID: \"eeccf333-5485-44f1-b0c6-617be538c81b\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ncl65" Jul 03 10:06:52.939750 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.939588 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbxh6\" (UniqueName: \"kubernetes.io/projected/eeccf333-5485-44f1-b0c6-617be538c81b-kube-api-access-xbxh6\") pod \"monitoring-plugin-d5dcff7cf-ncl65\" (UID: \"eeccf333-5485-44f1-b0c6-617be538c81b\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ncl65" Jul 03 10:06:52.939750 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.939618 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/67615ad3-a3cc-45e9-a148-c0476d653e11-metrics-server-audit-profiles\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.939750 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.939634 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6579\" (UniqueName: \"kubernetes.io/projected/67615ad3-a3cc-45e9-a148-c0476d653e11-kube-api-access-s6579\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.939750 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.939666 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/67615ad3-a3cc-45e9-a148-c0476d653e11-audit-log\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.939750 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.939694 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/67615ad3-a3cc-45e9-a148-c0476d653e11-secret-metrics-server-client-certs\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.940059 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.939753 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/67615ad3-a3cc-45e9-a148-c0476d653e11-secret-metrics-server-tls\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.940059 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.939778 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67615ad3-a3cc-45e9-a148-c0476d653e11-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.940059 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.939802 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/67615ad3-a3cc-45e9-a148-c0476d653e11-client-ca-bundle\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.940339 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.940309 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/67615ad3-a3cc-45e9-a148-c0476d653e11-audit-log\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.940790 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.940726 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67615ad3-a3cc-45e9-a148-c0476d653e11-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.940899 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.940842 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/67615ad3-a3cc-45e9-a148-c0476d653e11-metrics-server-audit-profiles\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.942895 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.942856 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/67615ad3-a3cc-45e9-a148-c0476d653e11-secret-metrics-server-client-certs\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.943006 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.942899 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eeccf333-5485-44f1-b0c6-617be538c81b-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-ncl65\" (UID: \"eeccf333-5485-44f1-b0c6-617be538c81b\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ncl65" Jul 03 10:06:52.943166 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.943143 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/67615ad3-a3cc-45e9-a148-c0476d653e11-secret-metrics-server-tls\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.943324 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.943307 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/67615ad3-a3cc-45e9-a148-c0476d653e11-client-ca-bundle\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.969410 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.969383 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6579\" (UniqueName: \"kubernetes.io/projected/67615ad3-a3cc-45e9-a148-c0476d653e11-kube-api-access-s6579\") pod \"metrics-server-865d4c69d7-pppdd\" (UID: \"67615ad3-a3cc-45e9-a148-c0476d653e11\") " pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:52.971595 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.971566 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbxh6\" (UniqueName: \"kubernetes.io/projected/eeccf333-5485-44f1-b0c6-617be538c81b-kube-api-access-xbxh6\") pod \"monitoring-plugin-d5dcff7cf-ncl65\" (UID: \"eeccf333-5485-44f1-b0c6-617be538c81b\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ncl65" Jul 03 10:06:52.978246 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:52.978210 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:06:53.031703 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.031668 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ncl65" Jul 03 10:06:53.040302 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.040259 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8q6k\" (UniqueName: \"kubernetes.io/projected/da235739-fe1e-4e3a-a2d4-e6d786071b13-kube-api-access-l8q6k\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.040688 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.040316 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da235739-fe1e-4e3a-a2d4-e6d786071b13-console-serving-cert\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.040688 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.040355 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-service-ca\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.040688 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.040421 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-oauth-serving-cert\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.040688 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.040469 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-trusted-ca-bundle\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.040688 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.040518 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da235739-fe1e-4e3a-a2d4-e6d786071b13-console-oauth-config\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.040688 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.040586 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-console-config\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.141687 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.141642 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-trusted-ca-bundle\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.141893 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.141699 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da235739-fe1e-4e3a-a2d4-e6d786071b13-console-oauth-config\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.141893 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.141771 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-console-config\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.141893 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.141818 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8q6k\" (UniqueName: \"kubernetes.io/projected/da235739-fe1e-4e3a-a2d4-e6d786071b13-kube-api-access-l8q6k\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.141893 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.141846 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da235739-fe1e-4e3a-a2d4-e6d786071b13-console-serving-cert\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.142101 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.141911 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-service-ca\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.142101 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.141955 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-oauth-serving-cert\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.142694 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.142670 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-trusted-ca-bundle\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.142889 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.142840 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-oauth-serving-cert\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.142889 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.142884 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-console-config\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.143068 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.142965 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-service-ca\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.144574 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.144551 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da235739-fe1e-4e3a-a2d4-e6d786071b13-console-oauth-config\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.144840 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.144815 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da235739-fe1e-4e3a-a2d4-e6d786071b13-console-serving-cert\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.153402 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.153382 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8q6k\" (UniqueName: \"kubernetes.io/projected/da235739-fe1e-4e3a-a2d4-e6d786071b13-kube-api-access-l8q6k\") pod \"console-77849d98d-pxzz4\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.206455 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.206381 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:06:53.361555 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:53.361525 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7f8d657f7c-j76pt"] Jul 03 10:06:54.227719 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.227688 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 03 10:06:54.238317 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.238293 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.242823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.242800 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 03 10:06:54.243075 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.243058 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 03 10:06:54.243153 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.243135 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 03 10:06:54.244211 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.244192 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 03 10:06:54.244323 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.244199 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-m4p7n\"" Jul 03 10:06:54.244323 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.244318 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 03 10:06:54.244640 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.244622 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cb1190dg0om3g\"" Jul 03 10:06:54.244754 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.244676 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 03 10:06:54.245355 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.245333 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 03 10:06:54.245462 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.245364 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 03 10:06:54.245535 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.245339 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 03 10:06:54.245535 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.245497 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 03 10:06:54.245747 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.245710 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 03 10:06:54.247096 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.246012 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 03 10:06:54.248219 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.248199 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 03 10:06:54.250607 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.250589 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 03 10:06:54.251177 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.251158 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 03 10:06:54.293541 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:06:54.293501 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92357b58_c179_4495_be4f_295b61cc0ad5.slice/crio-82a04b84ea527ec5a61e75075824c3a32267d02b90b7a3cf9ed104831708e7f1 WatchSource:0}: Error finding container 82a04b84ea527ec5a61e75075824c3a32267d02b90b7a3cf9ed104831708e7f1: Status 404 returned error can't find the container with id 82a04b84ea527ec5a61e75075824c3a32267d02b90b7a3cf9ed104831708e7f1 Jul 03 10:06:54.355320 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.354435 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.355320 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.354500 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.355320 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.354536 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.355320 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.354565 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.355320 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.354592 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.355320 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.354618 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.355320 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.354656 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af65bbd0-8f4a-48a1-82e5-97790033560b-config-out\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.355320 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.354686 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.355320 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.354714 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.355320 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.354758 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-web-config\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.355320 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.354781 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-config\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.355320 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.354803 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af65bbd0-8f4a-48a1-82e5-97790033560b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.355320 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.354854 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.356015 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.354877 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.356015 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.354913 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.356015 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.354938 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.356015 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.354975 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp95r\" (UniqueName: \"kubernetes.io/projected/af65bbd0-8f4a-48a1-82e5-97790033560b-kube-api-access-rp95r\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.356015 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.355003 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.356015 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.355049 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.356015 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.355078 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.476065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.455856 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af65bbd0-8f4a-48a1-82e5-97790033560b-config-out\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.476065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.455907 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.476065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.455940 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.476065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.455969 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-web-config\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.476065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.455999 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-config\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.476065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.456022 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af65bbd0-8f4a-48a1-82e5-97790033560b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.476065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.456064 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.476065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.456094 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.476065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.456132 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.476065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.456183 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.476065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.456220 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp95r\" (UniqueName: \"kubernetes.io/projected/af65bbd0-8f4a-48a1-82e5-97790033560b-kube-api-access-rp95r\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.476065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.456247 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.476065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.456288 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.476065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.456315 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.483337 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.456369 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.483337 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.456399 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.483337 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.456433 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.483337 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.456461 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.483337 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.456494 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.483337 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.456525 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.483337 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.464944 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.483337 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.470517 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.483337 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.471081 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.483337 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.474163 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.483337 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.474980 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.483337 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.475220 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.483337 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.481890 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-web-config\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.492879 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.484874 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.492879 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.486522 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.492879 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.488554 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.492879 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.489209 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.492879 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.489608 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af65bbd0-8f4a-48a1-82e5-97790033560b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.492879 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.489719 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.492879 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.490104 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.492879 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.490698 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.492879 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.491720 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.492879 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.491992 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af65bbd0-8f4a-48a1-82e5-97790033560b-config-out\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.492879 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.492152 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-config\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.492879 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.492783 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.492879 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.492824 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp95r\" (UniqueName: \"kubernetes.io/projected/af65bbd0-8f4a-48a1-82e5-97790033560b-kube-api-access-rp95r\") pod \"prometheus-k8s-0\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.538983 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.538105 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kxh5b"] Jul 03 10:06:54.540412 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:06:54.540206 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76d0eec5_d0d7_4cf4_9229_b3a930c68566.slice/crio-08b721fd8b9f4fbde28760924a87c7e51145d02ead0e13a78925ebca972c58a5 WatchSource:0}: Error finding container 08b721fd8b9f4fbde28760924a87c7e51145d02ead0e13a78925ebca972c58a5: Status 404 returned error can't find the container with id 08b721fd8b9f4fbde28760924a87c7e51145d02ead0e13a78925ebca972c58a5 Jul 03 10:06:54.552261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.552067 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:06:54.591151 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.590601 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-865d4c69d7-pppdd"] Jul 03 10:06:54.603454 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.597584 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-ncl65"] Jul 03 10:06:54.603454 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.602788 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77849d98d-pxzz4"] Jul 03 10:06:54.632308 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.632203 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qqgm8"] Jul 03 10:06:54.648570 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.646832 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ffkpf"] Jul 03 10:06:54.793362 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:54.793290 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 03 10:06:54.797567 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:06:54.797538 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf65bbd0_8f4a_48a1_82e5_97790033560b.slice/crio-0b36b20a856613e04c88ccecac7499ad6e9763c285abe9edde4f0958ffc05e43 WatchSource:0}: Error finding container 0b36b20a856613e04c88ccecac7499ad6e9763c285abe9edde4f0958ffc05e43: Status 404 returned error can't find the container with id 0b36b20a856613e04c88ccecac7499ad6e9763c285abe9edde4f0958ffc05e43 Jul 03 10:06:55.047149 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.047111 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gfjrs" event={"ID":"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318","Type":"ContainerStarted","Data":"fcc25ffae9231541bb07cb3c05a91e1fb4f88d268d23e2cc80a937aabe7cf566"} Jul 03 10:06:55.047299 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.047184 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gfjrs" event={"ID":"cdeaefc1-cb0f-4c41-8e6d-14918dfe9318","Type":"ContainerStarted","Data":"90730d27c7ffd2838bc555c3c6c7c6f436f904397367c438b6fcb603cb6591c0"} Jul 03 10:06:55.048654 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.048630 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857f869b78-dpzgg" event={"ID":"86c2c605-065a-454b-bcb2-5cc057993daf","Type":"ContainerStarted","Data":"a3f5c8576a1e6a1954fd60f39920ac851fc5bf3001355a431bf6bdec86cdcd2f"} Jul 03 10:06:55.049790 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.049767 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" event={"ID":"67615ad3-a3cc-45e9-a148-c0476d653e11","Type":"ContainerStarted","Data":"587b0c21e6cd1100d8dd8e0eb5c413dc046d23985c9ede6a2604fd62d3329ec4"} Jul 03 10:06:55.050881 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.050857 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" event={"ID":"92357b58-c179-4495-be4f-295b61cc0ad5","Type":"ContainerStarted","Data":"82a04b84ea527ec5a61e75075824c3a32267d02b90b7a3cf9ed104831708e7f1"} Jul 03 10:06:55.053271 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.053247 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n" event={"ID":"8f73dc61-257e-4a34-80ef-8c86c5d4511e","Type":"ContainerStarted","Data":"f8db9e8eda979b0edc2e03867be433897b042ea946a3224b944b803bd01a1f41"} Jul 03 10:06:55.055293 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.055270 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" event={"ID":"7b2ebc75-c820-4e81-9704-e2580f24da82","Type":"ContainerStarted","Data":"74b1f1a3f5a4b2b4226b63446a637356b8cbd71586b97a6a2feabd43c2267461"} Jul 03 10:06:55.055398 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.055326 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" event={"ID":"7b2ebc75-c820-4e81-9704-e2580f24da82","Type":"ContainerStarted","Data":"6091740653c5549597b18922dba51ef13b6ca7d3b927e5ea1965211e3252f733"} Jul 03 10:06:55.055398 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.055365 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" event={"ID":"7b2ebc75-c820-4e81-9704-e2580f24da82","Type":"ContainerStarted","Data":"51a4f62436af4fa37d3377b9082639462ca64b0efa5081ff77f497140794cbbd"} Jul 03 10:06:55.056715 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.056693 2573 generic.go:356] "Generic (PLEG): container finished" podID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerID="b5276ec3e3ce9a937da4598ddc3877e67d75cd757c875a65d8b4d39a062c49a4" exitCode=0 Jul 03 10:06:55.056814 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.056762 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af65bbd0-8f4a-48a1-82e5-97790033560b","Type":"ContainerDied","Data":"b5276ec3e3ce9a937da4598ddc3877e67d75cd757c875a65d8b4d39a062c49a4"} Jul 03 10:06:55.056861 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.056831 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af65bbd0-8f4a-48a1-82e5-97790033560b","Type":"ContainerStarted","Data":"0b36b20a856613e04c88ccecac7499ad6e9763c285abe9edde4f0958ffc05e43"} Jul 03 10:06:55.057868 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.057840 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ffkpf" event={"ID":"a077a8e0-9998-4603-9c7c-f92f296c2dbf","Type":"ContainerStarted","Data":"e7ba858180f3674f2ca6893c34db0a59be57b1fe572d964a11d4ed1b8755cd77"} Jul 03 10:06:55.059007 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.058984 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kxh5b" event={"ID":"76d0eec5-d0d7-4cf4-9229-b3a930c68566","Type":"ContainerStarted","Data":"08b721fd8b9f4fbde28760924a87c7e51145d02ead0e13a78925ebca972c58a5"} Jul 03 10:06:55.063379 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.063357 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bkrdf" event={"ID":"ebb4964b-254e-407e-b5b1-889066572d2d","Type":"ContainerStarted","Data":"525609418ab898c0944a52643f5454a6789f6d564c8c53504d3558816249cf17"} Jul 03 10:06:55.065544 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.065519 2573 generic.go:356] "Generic (PLEG): container finished" podID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerID="e98bc6ee39b35c913aec207246fffbeb9b61d36851bce552490fcd73d7efc301" exitCode=0 Jul 03 10:06:55.065649 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.065598 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4615fd38-b169-4b84-90f0-ca1ab940acd5","Type":"ContainerDied","Data":"e98bc6ee39b35c913aec207246fffbeb9b61d36851bce552490fcd73d7efc301"} Jul 03 10:06:55.067139 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.067117 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qqgm8" event={"ID":"e8ff375c-d36d-431c-ac6e-c2aef1d3c26a","Type":"ContainerStarted","Data":"ddeabb33b0625a55da0d9c3fb732425a286111e93fd8e2c1fdc4c4c6ac0d4f35"} Jul 03 10:06:55.069754 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.069711 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77849d98d-pxzz4" event={"ID":"da235739-fe1e-4e3a-a2d4-e6d786071b13","Type":"ContainerStarted","Data":"e132c3d30ed4a7eb021722f555754b875008813ee37a71bffd78a69ff8dc9d60"} Jul 03 10:06:55.069884 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.069868 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77849d98d-pxzz4" event={"ID":"da235739-fe1e-4e3a-a2d4-e6d786071b13","Type":"ContainerStarted","Data":"b9949b15191a85ba25abdcebd9c36e1a9bd2f715a1f33878ccbc31466f1a9d10"} Jul 03 10:06:55.071104 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.071083 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ncl65" event={"ID":"eeccf333-5485-44f1-b0c6-617be538c81b","Type":"ContainerStarted","Data":"b55d9ce2521aef32321b8383446872b3b62b0edf8a3795d4c89c0f4d2b949ad0"} Jul 03 10:06:55.072836 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.072782 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gfjrs" podStartSLOduration=5.840931294 podStartE2EDuration="7.072765798s" totalImagesPullingTime="1.141390438s" totalInitContainerRuntime="90.444066ms" isStatefulPod=true podCreationTimestamp="2026-07-03 10:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:55.072225028 +0000 UTC m=+69.984114324" watchObservedRunningTime="2026-07-03 10:06:55.072765798 +0000 UTC m=+69.984655102" Jul 03 10:06:55.107836 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.107780 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bkrdf" podStartSLOduration=2.398175537 podStartE2EDuration="12.107759607s" totalImagesPullingTime="9.70958407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:06:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:55.107659917 +0000 UTC m=+70.019549223" watchObservedRunningTime="2026-07-03 10:06:55.107759607 +0000 UTC m=+70.019648908" Jul 03 10:06:55.156533 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.156463 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-857f869b78-dpzgg" podStartSLOduration=4.764683563 podStartE2EDuration="9.156442255s" totalImagesPullingTime="4.391758692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:06:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:55.155582051 +0000 UTC m=+70.067471357" watchObservedRunningTime="2026-07-03 10:06:55.156442255 +0000 UTC m=+70.068331630" Jul 03 10:06:55.277696 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.277579 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-7gn6n" podStartSLOduration=4.081004436 podStartE2EDuration="8.277555158s" totalImagesPullingTime="4.196550722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:06:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:55.275277697 +0000 UTC m=+70.187167002" watchObservedRunningTime="2026-07-03 10:06:55.277555158 +0000 UTC m=+70.189444462" Jul 03 10:06:55.336543 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.335126 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-vr6b4" podStartSLOduration=2.993828702 podStartE2EDuration="7.335099137s" totalImagesPullingTime="4.341270435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:55.302571769 +0000 UTC m=+70.214461077" watchObservedRunningTime="2026-07-03 10:06:55.335099137 +0000 UTC m=+70.246988645" Jul 03 10:06:55.336543 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:55.335893 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77849d98d-pxzz4" podStartSLOduration=3.335881318 podStartE2EDuration="3.335881318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:06:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:06:55.33409683 +0000 UTC m=+70.245986134" watchObservedRunningTime="2026-07-03 10:06:55.335881318 +0000 UTC m=+70.247770604" Jul 03 10:06:57.290440 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:57.290404 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:57.291864 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:57.291844 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:57.298654 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:57.298629 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:58.094250 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:58.094040 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:06:59.040627 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:06:59.040537 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-x27kq" Jul 03 10:07:03.117599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.117554 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" event={"ID":"67615ad3-a3cc-45e9-a148-c0476d653e11","Type":"ContainerStarted","Data":"7396d8f5e27e0cad0fa617ba645234962bc878888070a07d6ec13332db20ac9f"} Jul 03 10:07:03.121353 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.121180 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" event={"ID":"92357b58-c179-4495-be4f-295b61cc0ad5","Type":"ContainerStarted","Data":"d974ca8eff526ad577b779beb0ca1ffd4fbe780738408d3fe49dae93f6b72f7e"} Jul 03 10:07:03.121353 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.121259 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" event={"ID":"92357b58-c179-4495-be4f-295b61cc0ad5","Type":"ContainerStarted","Data":"04e31e8ad03c38dea976a5e661f0b700ebf6fd13a30a61ef8383203c4959ef84"} Jul 03 10:07:03.121353 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.121305 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" event={"ID":"92357b58-c179-4495-be4f-295b61cc0ad5","Type":"ContainerStarted","Data":"14ad83d5f8a476551919c0f57b3e009d5017f2a15a71a792b2dc557f2163a3ab"} Jul 03 10:07:03.126851 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.126817 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af65bbd0-8f4a-48a1-82e5-97790033560b","Type":"ContainerStarted","Data":"4faf9001514fd18bdf6596ce0cb7e031203808206aecc20e0a625dac4dee090a"} Jul 03 10:07:03.126982 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.126915 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af65bbd0-8f4a-48a1-82e5-97790033560b","Type":"ContainerStarted","Data":"2cfa98fa7d32e377f93c19770c62cedc6312e76c59e422d938ec18b0e5015c67"} Jul 03 10:07:03.127044 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.126989 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af65bbd0-8f4a-48a1-82e5-97790033560b","Type":"ContainerStarted","Data":"129b775dda6ab6865eb8e22d49468371b16c90e621bdb153f9f5bf17b13b7150"} Jul 03 10:07:03.127096 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.127053 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af65bbd0-8f4a-48a1-82e5-97790033560b","Type":"ContainerStarted","Data":"568ba03def1e9590d1e39fda5b1dafc85c2146aaafcd946e6b29a051f5048736"} Jul 03 10:07:03.127148 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.127095 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af65bbd0-8f4a-48a1-82e5-97790033560b","Type":"ContainerStarted","Data":"2c1f72b88a783a62fcd60dd0bbdcb528b54cddb1937e05cb1e43375bdc2ddc4c"} Jul 03 10:07:03.127200 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.127166 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af65bbd0-8f4a-48a1-82e5-97790033560b","Type":"ContainerStarted","Data":"a0f6f8052fb1b3ea367b51ff7eddb4dcbecb0cc48ade08978305ff785a65f263"} Jul 03 10:07:03.129245 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.129219 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ffkpf" event={"ID":"a077a8e0-9998-4603-9c7c-f92f296c2dbf","Type":"ContainerStarted","Data":"ee5471e2f8031c9add415b2c7ca10d554f78d170d3fdfd4b50602d391a9bac91"} Jul 03 10:07:03.129349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.129268 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ffkpf" event={"ID":"a077a8e0-9998-4603-9c7c-f92f296c2dbf","Type":"ContainerStarted","Data":"cbf112f414efc9f1aaf134d0fb02e6465aa36c078a13414e9517be79a8df5ca5"} Jul 03 10:07:03.132945 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.132918 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kxh5b" event={"ID":"76d0eec5-d0d7-4cf4-9229-b3a930c68566","Type":"ContainerStarted","Data":"b2255e345a553093c0f1d133505eedd4afb32d299e6c7f25888303681df50bf2"} Jul 03 10:07:03.133055 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.133027 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:07:03.137159 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.137127 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4615fd38-b169-4b84-90f0-ca1ab940acd5","Type":"ContainerStarted","Data":"9984de009afe019cf8d880815f719c17c5b00ae5469652993f6daaa97ecb548e"} Jul 03 10:07:03.137256 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.137196 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4615fd38-b169-4b84-90f0-ca1ab940acd5","Type":"ContainerStarted","Data":"7567e46de7eb1fc4521a7e18419d724780a3b7b5d07708e6797051df130f6964"} Jul 03 10:07:03.137256 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.137239 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4615fd38-b169-4b84-90f0-ca1ab940acd5","Type":"ContainerStarted","Data":"59eec91e37756213422616f31141904928bb1589d63159cf247432d5021600e7"} Jul 03 10:07:03.137329 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.137281 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4615fd38-b169-4b84-90f0-ca1ab940acd5","Type":"ContainerStarted","Data":"47fca4da61224c44d2aec55d3dc9003d5ac0fff33f0a6232cee720c76001930a"} Jul 03 10:07:03.137329 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.137321 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4615fd38-b169-4b84-90f0-ca1ab940acd5","Type":"ContainerStarted","Data":"55385124e578d5e7164e179d51e2ffde7e82d3d374fe0c88c5d95fa3adfc6d7c"} Jul 03 10:07:03.139086 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.139064 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qqgm8" event={"ID":"e8ff375c-d36d-431c-ac6e-c2aef1d3c26a","Type":"ContainerStarted","Data":"351110f05edc070b68da504117aaa96d9bb09f83c56ed48e4c8ca96838ea29c6"} Jul 03 10:07:03.141763 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.141212 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ncl65" event={"ID":"eeccf333-5485-44f1-b0c6-617be538c81b","Type":"ContainerStarted","Data":"013c1e01b3c7751416da12d5121bf1b8c314710a4adbf4567cccc9cae8bd525c"} Jul 03 10:07:03.141862 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.141790 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ncl65" Jul 03 10:07:03.148946 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.148903 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ncl65" Jul 03 10:07:03.160068 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.160007 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" podStartSLOduration=3.716602939 podStartE2EDuration="11.159989913s" totalImagesPullingTime="7.443386974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:06:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:07:03.15864546 +0000 UTC m=+78.070534765" watchObservedRunningTime="2026-07-03 10:07:03.159989913 +0000 UTC m=+78.071879199" Jul 03 10:07:03.207300 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.207267 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:07:03.207476 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.207332 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:07:03.213330 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.213303 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:07:03.235699 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.235633 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kxh5b" podStartSLOduration=69.652114696 podStartE2EDuration="1m17.235615716s" totalImagesPullingTime="7.58350102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:05:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:07:03.233455322 +0000 UTC m=+78.145344626" watchObservedRunningTime="2026-07-03 10:07:03.235615716 +0000 UTC m=+78.147505020" Jul 03 10:07:03.236078 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.236052 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ffkpf" podStartSLOduration=69.807825164 podStartE2EDuration="1m17.236041189s" totalImagesPullingTime="7.428216025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:05:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:07:03.200206358 +0000 UTC m=+78.112095663" watchObservedRunningTime="2026-07-03 10:07:03.236041189 +0000 UTC m=+78.147930544" Jul 03 10:07:03.269559 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.269503 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qqgm8" podStartSLOduration=69.769959191 podStartE2EDuration="1m17.269485582s" totalImagesPullingTime="7.499526391s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:05:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:07:03.264256355 +0000 UTC m=+78.176145661" watchObservedRunningTime="2026-07-03 10:07:03.269485582 +0000 UTC m=+78.181374897" Jul 03 10:07:03.317225 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.317171 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ncl65" podStartSLOduration=3.869138557 podStartE2EDuration="11.317154758s" totalImagesPullingTime="7.448016201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:06:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:07:03.312834657 +0000 UTC m=+78.224723961" watchObservedRunningTime="2026-07-03 10:07:03.317154758 +0000 UTC m=+78.229044062" Jul 03 10:07:03.386306 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:03.386256 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.291216924 podStartE2EDuration="9.386240045s" totalImagesPullingTime="7.06835345s" totalInitContainerRuntime="26.669671ms" isStatefulPod=false podCreationTimestamp="2026-07-03 10:06:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:07:03.38454791 +0000 UTC m=+78.296437214" watchObservedRunningTime="2026-07-03 10:07:03.386240045 +0000 UTC m=+78.298129361" Jul 03 10:07:04.148421 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:04.148388 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" event={"ID":"92357b58-c179-4495-be4f-295b61cc0ad5","Type":"ContainerStarted","Data":"e6d84fee3e6ad87711bf64fc9687705fbceea50e9d0055cef575ad895c5c14d9"} Jul 03 10:07:04.148927 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:04.148511 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:07:04.148927 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:04.148547 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" event={"ID":"92357b58-c179-4495-be4f-295b61cc0ad5","Type":"ContainerStarted","Data":"7cffc4f4531b027da73a5b1029b40ce54f511bd6020914e3ef766b7aa497bdf5"} Jul 03 10:07:04.148927 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:04.148588 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" event={"ID":"92357b58-c179-4495-be4f-295b61cc0ad5","Type":"ContainerStarted","Data":"9aec36767122d680a32eed29a3e8bbcf67838cfbf003426ed9f78122b167b7d6"} Jul 03 10:07:04.151941 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:04.151908 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4615fd38-b169-4b84-90f0-ca1ab940acd5","Type":"ContainerStarted","Data":"e802960617bfb96115f6f71e9ff094bc4306fc87574d5265a5b6447ac3503658"} Jul 03 10:07:04.156387 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:04.156366 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:07:04.300615 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:04.300554 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" podStartSLOduration=5.676206587 podStartE2EDuration="14.300535153s" totalImagesPullingTime="8.624328566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:06:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:07:04.241925885 +0000 UTC m=+79.153815200" watchObservedRunningTime="2026-07-03 10:07:04.300535153 +0000 UTC m=+79.212424500" Jul 03 10:07:04.301640 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:04.301607 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.274008344 podStartE2EDuration="16.301595589s" totalImagesPullingTime="11.979691615s" totalInitContainerRuntime="47.89563ms" isStatefulPod=false podCreationTimestamp="2026-07-03 10:06:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:07:04.300367286 +0000 UTC m=+79.212256591" watchObservedRunningTime="2026-07-03 10:07:04.301595589 +0000 UTC m=+79.213484903" Jul 03 10:07:04.411492 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:04.411409 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-857f869b78-dpzgg"] Jul 03 10:07:04.552629 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:04.552580 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:07:10.162396 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:10.162285 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7f8d657f7c-j76pt" Jul 03 10:07:12.979008 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:12.978977 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:07:12.979375 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:12.979037 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:07:29.437274 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.437219 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-857f869b78-dpzgg" podUID="86c2c605-065a-454b-bcb2-5cc057993daf" containerName="console" containerID="cri-o://a3f5c8576a1e6a1954fd60f39920ac851fc5bf3001355a431bf6bdec86cdcd2f" gracePeriod=15 Jul 03 10:07:29.674877 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.674852 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:07:29.784269 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.784176 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-console-config\") pod \"86c2c605-065a-454b-bcb2-5cc057993daf\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " Jul 03 10:07:29.784269 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.784219 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-trusted-ca-bundle\") pod \"86c2c605-065a-454b-bcb2-5cc057993daf\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " Jul 03 10:07:29.784269 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.784241 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-oauth-serving-cert\") pod \"86c2c605-065a-454b-bcb2-5cc057993daf\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " Jul 03 10:07:29.784269 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.784264 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-service-ca\") pod \"86c2c605-065a-454b-bcb2-5cc057993daf\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " Jul 03 10:07:29.784574 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.784299 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86c2c605-065a-454b-bcb2-5cc057993daf-console-serving-cert\") pod \"86c2c605-065a-454b-bcb2-5cc057993daf\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " Jul 03 10:07:29.784574 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.784325 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86c2c605-065a-454b-bcb2-5cc057993daf-console-oauth-config\") pod \"86c2c605-065a-454b-bcb2-5cc057993daf\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " Jul 03 10:07:29.784574 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.784399 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2g22\" (UniqueName: \"kubernetes.io/projected/86c2c605-065a-454b-bcb2-5cc057993daf-kube-api-access-q2g22\") pod \"86c2c605-065a-454b-bcb2-5cc057993daf\" (UID: \"86c2c605-065a-454b-bcb2-5cc057993daf\") " Jul 03 10:07:29.784724 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.784619 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-service-ca" (OuterVolumeSpecName: "service-ca") pod "86c2c605-065a-454b-bcb2-5cc057993daf" (UID: "86c2c605-065a-454b-bcb2-5cc057993daf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:07:29.784825 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.784674 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-console-config" (OuterVolumeSpecName: "console-config") pod "86c2c605-065a-454b-bcb2-5cc057993daf" (UID: "86c2c605-065a-454b-bcb2-5cc057993daf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:07:29.784948 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.784916 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "86c2c605-065a-454b-bcb2-5cc057993daf" (UID: "86c2c605-065a-454b-bcb2-5cc057993daf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:07:29.785118 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.785000 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "86c2c605-065a-454b-bcb2-5cc057993daf" (UID: "86c2c605-065a-454b-bcb2-5cc057993daf"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:07:29.786860 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.786835 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86c2c605-065a-454b-bcb2-5cc057993daf-kube-api-access-q2g22" (OuterVolumeSpecName: "kube-api-access-q2g22") pod "86c2c605-065a-454b-bcb2-5cc057993daf" (UID: "86c2c605-065a-454b-bcb2-5cc057993daf"). InnerVolumeSpecName "kube-api-access-q2g22". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:07:29.786948 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.786836 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86c2c605-065a-454b-bcb2-5cc057993daf-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "86c2c605-065a-454b-bcb2-5cc057993daf" (UID: "86c2c605-065a-454b-bcb2-5cc057993daf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:07:29.786948 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.786908 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86c2c605-065a-454b-bcb2-5cc057993daf-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "86c2c605-065a-454b-bcb2-5cc057993daf" (UID: "86c2c605-065a-454b-bcb2-5cc057993daf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:07:29.886016 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.885975 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-console-config\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:07:29.886016 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.886005 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-trusted-ca-bundle\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:07:29.886016 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.886014 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-oauth-serving-cert\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:07:29.886016 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.886023 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/86c2c605-065a-454b-bcb2-5cc057993daf-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:07:29.886276 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.886032 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/86c2c605-065a-454b-bcb2-5cc057993daf-console-serving-cert\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:07:29.886276 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.886040 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/86c2c605-065a-454b-bcb2-5cc057993daf-console-oauth-config\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:07:29.886276 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:29.886048 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2g22\" (UniqueName: \"kubernetes.io/projected/86c2c605-065a-454b-bcb2-5cc057993daf-kube-api-access-q2g22\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:07:30.256440 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:30.256401 2573 generic.go:356] "Generic (PLEG): container finished" podID="86c2c605-065a-454b-bcb2-5cc057993daf" containerID="a3f5c8576a1e6a1954fd60f39920ac851fc5bf3001355a431bf6bdec86cdcd2f" exitCode=0 Jul 03 10:07:30.256610 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:30.256459 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857f869b78-dpzgg" Jul 03 10:07:30.256610 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:30.256480 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857f869b78-dpzgg" event={"ID":"86c2c605-065a-454b-bcb2-5cc057993daf","Type":"ContainerDied","Data":"a3f5c8576a1e6a1954fd60f39920ac851fc5bf3001355a431bf6bdec86cdcd2f"} Jul 03 10:07:30.256610 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:30.256563 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857f869b78-dpzgg" event={"ID":"86c2c605-065a-454b-bcb2-5cc057993daf","Type":"ContainerDied","Data":"55bb102bbc60b47004563299026ab6c511efbff7155da6638c086da21ed1cf6e"} Jul 03 10:07:30.256610 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:30.256595 2573 scope.go:117] "RemoveContainer" containerID="a3f5c8576a1e6a1954fd60f39920ac851fc5bf3001355a431bf6bdec86cdcd2f" Jul 03 10:07:30.265063 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:30.265044 2573 scope.go:117] "RemoveContainer" containerID="a3f5c8576a1e6a1954fd60f39920ac851fc5bf3001355a431bf6bdec86cdcd2f" Jul 03 10:07:30.265332 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:07:30.265314 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3f5c8576a1e6a1954fd60f39920ac851fc5bf3001355a431bf6bdec86cdcd2f\": container with ID starting with a3f5c8576a1e6a1954fd60f39920ac851fc5bf3001355a431bf6bdec86cdcd2f not found: ID does not exist" containerID="a3f5c8576a1e6a1954fd60f39920ac851fc5bf3001355a431bf6bdec86cdcd2f" Jul 03 10:07:30.265402 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:30.265339 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3f5c8576a1e6a1954fd60f39920ac851fc5bf3001355a431bf6bdec86cdcd2f"} err="failed to get container status \"a3f5c8576a1e6a1954fd60f39920ac851fc5bf3001355a431bf6bdec86cdcd2f\": rpc error: code = NotFound desc = could not find container \"a3f5c8576a1e6a1954fd60f39920ac851fc5bf3001355a431bf6bdec86cdcd2f\": container with ID starting with a3f5c8576a1e6a1954fd60f39920ac851fc5bf3001355a431bf6bdec86cdcd2f not found: ID does not exist" Jul 03 10:07:30.279939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:30.279917 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-857f869b78-dpzgg"] Jul 03 10:07:30.287039 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:30.287019 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-857f869b78-dpzgg"] Jul 03 10:07:30.562422 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:30.562343 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86c2c605-065a-454b-bcb2-5cc057993daf" path="/var/lib/kubelet/pods/86c2c605-065a-454b-bcb2-5cc057993daf/volumes" Jul 03 10:07:32.983216 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:32.983187 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:07:32.986923 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:32.986897 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-865d4c69d7-pppdd" Jul 03 10:07:34.155163 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:34.155125 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kxh5b" Jul 03 10:07:54.553464 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:54.553416 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:07:54.582522 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:54.582490 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:07:55.372304 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:07:55.372277 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:08.369374 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:08.369337 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 03 10:08:08.369923 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:08.369621 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="alertmanager" containerID="cri-o://55385124e578d5e7164e179d51e2ffde7e82d3d374fe0c88c5d95fa3adfc6d7c" gracePeriod=120 Jul 03 10:08:08.369923 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:08.369682 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="kube-rbac-proxy-metric" containerID="cri-o://9984de009afe019cf8d880815f719c17c5b00ae5469652993f6daaa97ecb548e" gracePeriod=120 Jul 03 10:08:08.369923 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:08.369700 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="kube-rbac-proxy-web" containerID="cri-o://59eec91e37756213422616f31141904928bb1589d63159cf247432d5021600e7" gracePeriod=120 Jul 03 10:08:08.369923 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:08.369717 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="kube-rbac-proxy" containerID="cri-o://7567e46de7eb1fc4521a7e18419d724780a3b7b5d07708e6797051df130f6964" gracePeriod=120 Jul 03 10:08:08.369923 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:08.369769 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="config-reloader" containerID="cri-o://47fca4da61224c44d2aec55d3dc9003d5ac0fff33f0a6232cee720c76001930a" gracePeriod=120 Jul 03 10:08:08.369923 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:08.369789 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="prom-label-proxy" containerID="cri-o://e802960617bfb96115f6f71e9ff094bc4306fc87574d5265a5b6447ac3503658" gracePeriod=120 Jul 03 10:08:09.413115 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.413082 2573 generic.go:356] "Generic (PLEG): container finished" podID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerID="e802960617bfb96115f6f71e9ff094bc4306fc87574d5265a5b6447ac3503658" exitCode=0 Jul 03 10:08:09.413115 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.413108 2573 generic.go:356] "Generic (PLEG): container finished" podID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerID="9984de009afe019cf8d880815f719c17c5b00ae5469652993f6daaa97ecb548e" exitCode=0 Jul 03 10:08:09.413115 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.413116 2573 generic.go:356] "Generic (PLEG): container finished" podID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerID="7567e46de7eb1fc4521a7e18419d724780a3b7b5d07708e6797051df130f6964" exitCode=0 Jul 03 10:08:09.413115 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.413121 2573 generic.go:356] "Generic (PLEG): container finished" podID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerID="47fca4da61224c44d2aec55d3dc9003d5ac0fff33f0a6232cee720c76001930a" exitCode=0 Jul 03 10:08:09.413115 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.413126 2573 generic.go:356] "Generic (PLEG): container finished" podID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerID="55385124e578d5e7164e179d51e2ffde7e82d3d374fe0c88c5d95fa3adfc6d7c" exitCode=0 Jul 03 10:08:09.413608 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.413151 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4615fd38-b169-4b84-90f0-ca1ab940acd5","Type":"ContainerDied","Data":"e802960617bfb96115f6f71e9ff094bc4306fc87574d5265a5b6447ac3503658"} Jul 03 10:08:09.413608 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.413213 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4615fd38-b169-4b84-90f0-ca1ab940acd5","Type":"ContainerDied","Data":"9984de009afe019cf8d880815f719c17c5b00ae5469652993f6daaa97ecb548e"} Jul 03 10:08:09.413608 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.413239 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4615fd38-b169-4b84-90f0-ca1ab940acd5","Type":"ContainerDied","Data":"7567e46de7eb1fc4521a7e18419d724780a3b7b5d07708e6797051df130f6964"} Jul 03 10:08:09.413608 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.413264 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4615fd38-b169-4b84-90f0-ca1ab940acd5","Type":"ContainerDied","Data":"47fca4da61224c44d2aec55d3dc9003d5ac0fff33f0a6232cee720c76001930a"} Jul 03 10:08:09.413608 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.413290 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4615fd38-b169-4b84-90f0-ca1ab940acd5","Type":"ContainerDied","Data":"55385124e578d5e7164e179d51e2ffde7e82d3d374fe0c88c5d95fa3adfc6d7c"} Jul 03 10:08:09.616623 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.616590 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:09.739431 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.739336 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4615fd38-b169-4b84-90f0-ca1ab940acd5-config-out\") pod \"4615fd38-b169-4b84-90f0-ca1ab940acd5\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " Jul 03 10:08:09.739431 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.739395 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4615fd38-b169-4b84-90f0-ca1ab940acd5-alertmanager-main-db\") pod \"4615fd38-b169-4b84-90f0-ca1ab940acd5\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " Jul 03 10:08:09.739431 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.739416 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-kube-rbac-proxy-web\") pod \"4615fd38-b169-4b84-90f0-ca1ab940acd5\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " Jul 03 10:08:09.739431 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.739437 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-cluster-tls-config\") pod \"4615fd38-b169-4b84-90f0-ca1ab940acd5\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " Jul 03 10:08:09.739783 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.739526 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8c5f\" (UniqueName: \"kubernetes.io/projected/4615fd38-b169-4b84-90f0-ca1ab940acd5-kube-api-access-p8c5f\") pod \"4615fd38-b169-4b84-90f0-ca1ab940acd5\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " Jul 03 10:08:09.739783 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.739548 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-config-volume\") pod \"4615fd38-b169-4b84-90f0-ca1ab940acd5\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " Jul 03 10:08:09.739783 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.739574 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4615fd38-b169-4b84-90f0-ca1ab940acd5-alertmanager-trusted-ca-bundle\") pod \"4615fd38-b169-4b84-90f0-ca1ab940acd5\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " Jul 03 10:08:09.739783 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.739597 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-main-tls\") pod \"4615fd38-b169-4b84-90f0-ca1ab940acd5\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " Jul 03 10:08:09.739783 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.739620 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4615fd38-b169-4b84-90f0-ca1ab940acd5-metrics-client-ca\") pod \"4615fd38-b169-4b84-90f0-ca1ab940acd5\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " Jul 03 10:08:09.739783 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.739677 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4615fd38-b169-4b84-90f0-ca1ab940acd5-tls-assets\") pod \"4615fd38-b169-4b84-90f0-ca1ab940acd5\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " Jul 03 10:08:09.739783 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.739713 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"4615fd38-b169-4b84-90f0-ca1ab940acd5\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " Jul 03 10:08:09.739783 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.739758 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-web-config\") pod \"4615fd38-b169-4b84-90f0-ca1ab940acd5\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " Jul 03 10:08:09.739783 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.739762 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4615fd38-b169-4b84-90f0-ca1ab940acd5-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "4615fd38-b169-4b84-90f0-ca1ab940acd5" (UID: "4615fd38-b169-4b84-90f0-ca1ab940acd5"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:08:09.739783 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.739783 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-kube-rbac-proxy\") pod \"4615fd38-b169-4b84-90f0-ca1ab940acd5\" (UID: \"4615fd38-b169-4b84-90f0-ca1ab940acd5\") " Jul 03 10:08:09.740387 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.740150 2573 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4615fd38-b169-4b84-90f0-ca1ab940acd5-alertmanager-main-db\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:09.740387 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.740231 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4615fd38-b169-4b84-90f0-ca1ab940acd5-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "4615fd38-b169-4b84-90f0-ca1ab940acd5" (UID: "4615fd38-b169-4b84-90f0-ca1ab940acd5"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:08:09.740532 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.740506 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4615fd38-b169-4b84-90f0-ca1ab940acd5-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "4615fd38-b169-4b84-90f0-ca1ab940acd5" (UID: "4615fd38-b169-4b84-90f0-ca1ab940acd5"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:08:09.742578 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.742532 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4615fd38-b169-4b84-90f0-ca1ab940acd5-kube-api-access-p8c5f" (OuterVolumeSpecName: "kube-api-access-p8c5f") pod "4615fd38-b169-4b84-90f0-ca1ab940acd5" (UID: "4615fd38-b169-4b84-90f0-ca1ab940acd5"). InnerVolumeSpecName "kube-api-access-p8c5f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:08:09.742705 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.742626 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4615fd38-b169-4b84-90f0-ca1ab940acd5-config-out" (OuterVolumeSpecName: "config-out") pod "4615fd38-b169-4b84-90f0-ca1ab940acd5" (UID: "4615fd38-b169-4b84-90f0-ca1ab940acd5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:08:09.743116 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.743088 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "4615fd38-b169-4b84-90f0-ca1ab940acd5" (UID: "4615fd38-b169-4b84-90f0-ca1ab940acd5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:08:09.743421 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.743286 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "4615fd38-b169-4b84-90f0-ca1ab940acd5" (UID: "4615fd38-b169-4b84-90f0-ca1ab940acd5"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:08:09.743421 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.743357 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "4615fd38-b169-4b84-90f0-ca1ab940acd5" (UID: "4615fd38-b169-4b84-90f0-ca1ab940acd5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:08:09.743998 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.743964 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "4615fd38-b169-4b84-90f0-ca1ab940acd5" (UID: "4615fd38-b169-4b84-90f0-ca1ab940acd5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:08:09.744138 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.744117 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-config-volume" (OuterVolumeSpecName: "config-volume") pod "4615fd38-b169-4b84-90f0-ca1ab940acd5" (UID: "4615fd38-b169-4b84-90f0-ca1ab940acd5"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:08:09.744198 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.744124 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4615fd38-b169-4b84-90f0-ca1ab940acd5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4615fd38-b169-4b84-90f0-ca1ab940acd5" (UID: "4615fd38-b169-4b84-90f0-ca1ab940acd5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:08:09.747926 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.747897 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "4615fd38-b169-4b84-90f0-ca1ab940acd5" (UID: "4615fd38-b169-4b84-90f0-ca1ab940acd5"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:08:09.754971 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.754909 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-web-config" (OuterVolumeSpecName: "web-config") pod "4615fd38-b169-4b84-90f0-ca1ab940acd5" (UID: "4615fd38-b169-4b84-90f0-ca1ab940acd5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:08:09.841500 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.841457 2573 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4615fd38-b169-4b84-90f0-ca1ab940acd5-tls-assets\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:09.841500 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.841491 2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:09.841500 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.841500 2573 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-web-config\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:09.841500 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.841509 2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:09.841790 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.841518 2573 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4615fd38-b169-4b84-90f0-ca1ab940acd5-config-out\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:09.841790 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.841526 2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:09.841790 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.841535 2573 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-cluster-tls-config\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:09.841790 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.841543 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8c5f\" (UniqueName: \"kubernetes.io/projected/4615fd38-b169-4b84-90f0-ca1ab940acd5-kube-api-access-p8c5f\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:09.841790 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.841551 2573 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-config-volume\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:09.841790 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.841559 2573 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4615fd38-b169-4b84-90f0-ca1ab940acd5-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:09.841790 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.841567 2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4615fd38-b169-4b84-90f0-ca1ab940acd5-secret-alertmanager-main-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:09.841790 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:09.841577 2573 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4615fd38-b169-4b84-90f0-ca1ab940acd5-metrics-client-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:10.420423 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.420393 2573 generic.go:356] "Generic (PLEG): container finished" podID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerID="59eec91e37756213422616f31141904928bb1589d63159cf247432d5021600e7" exitCode=0 Jul 03 10:08:10.420919 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.420462 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4615fd38-b169-4b84-90f0-ca1ab940acd5","Type":"ContainerDied","Data":"59eec91e37756213422616f31141904928bb1589d63159cf247432d5021600e7"} Jul 03 10:08:10.420919 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.420468 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.420919 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.420541 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4615fd38-b169-4b84-90f0-ca1ab940acd5","Type":"ContainerDied","Data":"db4216ee64485141d352f2b3e383eea2dda217a9bca0a87e5e18694a9e8bbd8b"} Jul 03 10:08:10.420919 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.420593 2573 scope.go:117] "RemoveContainer" containerID="e802960617bfb96115f6f71e9ff094bc4306fc87574d5265a5b6447ac3503658" Jul 03 10:08:10.435902 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.435880 2573 scope.go:117] "RemoveContainer" containerID="9984de009afe019cf8d880815f719c17c5b00ae5469652993f6daaa97ecb548e" Jul 03 10:08:10.442967 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.442952 2573 scope.go:117] "RemoveContainer" containerID="7567e46de7eb1fc4521a7e18419d724780a3b7b5d07708e6797051df130f6964" Jul 03 10:08:10.450511 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.450486 2573 scope.go:117] "RemoveContainer" containerID="59eec91e37756213422616f31141904928bb1589d63159cf247432d5021600e7" Jul 03 10:08:10.454681 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.454658 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 03 10:08:10.458998 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.458973 2573 scope.go:117] "RemoveContainer" containerID="47fca4da61224c44d2aec55d3dc9003d5ac0fff33f0a6232cee720c76001930a" Jul 03 10:08:10.459957 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.459905 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 03 10:08:10.466080 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.466061 2573 scope.go:117] "RemoveContainer" containerID="55385124e578d5e7164e179d51e2ffde7e82d3d374fe0c88c5d95fa3adfc6d7c" Jul 03 10:08:10.473051 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.473031 2573 scope.go:117] "RemoveContainer" containerID="e98bc6ee39b35c913aec207246fffbeb9b61d36851bce552490fcd73d7efc301" Jul 03 10:08:10.479513 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.479497 2573 scope.go:117] "RemoveContainer" containerID="e802960617bfb96115f6f71e9ff094bc4306fc87574d5265a5b6447ac3503658" Jul 03 10:08:10.479781 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:08:10.479762 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e802960617bfb96115f6f71e9ff094bc4306fc87574d5265a5b6447ac3503658\": container with ID starting with e802960617bfb96115f6f71e9ff094bc4306fc87574d5265a5b6447ac3503658 not found: ID does not exist" containerID="e802960617bfb96115f6f71e9ff094bc4306fc87574d5265a5b6447ac3503658" Jul 03 10:08:10.479835 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.479791 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e802960617bfb96115f6f71e9ff094bc4306fc87574d5265a5b6447ac3503658"} err="failed to get container status \"e802960617bfb96115f6f71e9ff094bc4306fc87574d5265a5b6447ac3503658\": rpc error: code = NotFound desc = could not find container \"e802960617bfb96115f6f71e9ff094bc4306fc87574d5265a5b6447ac3503658\": container with ID starting with e802960617bfb96115f6f71e9ff094bc4306fc87574d5265a5b6447ac3503658 not found: ID does not exist" Jul 03 10:08:10.479835 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.479809 2573 scope.go:117] "RemoveContainer" containerID="9984de009afe019cf8d880815f719c17c5b00ae5469652993f6daaa97ecb548e" Jul 03 10:08:10.480048 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:08:10.480034 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9984de009afe019cf8d880815f719c17c5b00ae5469652993f6daaa97ecb548e\": container with ID starting with 9984de009afe019cf8d880815f719c17c5b00ae5469652993f6daaa97ecb548e not found: ID does not exist" containerID="9984de009afe019cf8d880815f719c17c5b00ae5469652993f6daaa97ecb548e" Jul 03 10:08:10.480098 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.480051 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9984de009afe019cf8d880815f719c17c5b00ae5469652993f6daaa97ecb548e"} err="failed to get container status \"9984de009afe019cf8d880815f719c17c5b00ae5469652993f6daaa97ecb548e\": rpc error: code = NotFound desc = could not find container \"9984de009afe019cf8d880815f719c17c5b00ae5469652993f6daaa97ecb548e\": container with ID starting with 9984de009afe019cf8d880815f719c17c5b00ae5469652993f6daaa97ecb548e not found: ID does not exist" Jul 03 10:08:10.480098 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.480063 2573 scope.go:117] "RemoveContainer" containerID="7567e46de7eb1fc4521a7e18419d724780a3b7b5d07708e6797051df130f6964" Jul 03 10:08:10.480270 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:08:10.480254 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7567e46de7eb1fc4521a7e18419d724780a3b7b5d07708e6797051df130f6964\": container with ID starting with 7567e46de7eb1fc4521a7e18419d724780a3b7b5d07708e6797051df130f6964 not found: ID does not exist" containerID="7567e46de7eb1fc4521a7e18419d724780a3b7b5d07708e6797051df130f6964" Jul 03 10:08:10.480310 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.480275 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7567e46de7eb1fc4521a7e18419d724780a3b7b5d07708e6797051df130f6964"} err="failed to get container status \"7567e46de7eb1fc4521a7e18419d724780a3b7b5d07708e6797051df130f6964\": rpc error: code = NotFound desc = could not find container \"7567e46de7eb1fc4521a7e18419d724780a3b7b5d07708e6797051df130f6964\": container with ID starting with 7567e46de7eb1fc4521a7e18419d724780a3b7b5d07708e6797051df130f6964 not found: ID does not exist" Jul 03 10:08:10.480310 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.480289 2573 scope.go:117] "RemoveContainer" containerID="59eec91e37756213422616f31141904928bb1589d63159cf247432d5021600e7" Jul 03 10:08:10.480498 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:08:10.480480 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59eec91e37756213422616f31141904928bb1589d63159cf247432d5021600e7\": container with ID starting with 59eec91e37756213422616f31141904928bb1589d63159cf247432d5021600e7 not found: ID does not exist" containerID="59eec91e37756213422616f31141904928bb1589d63159cf247432d5021600e7" Jul 03 10:08:10.480537 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.480503 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59eec91e37756213422616f31141904928bb1589d63159cf247432d5021600e7"} err="failed to get container status \"59eec91e37756213422616f31141904928bb1589d63159cf247432d5021600e7\": rpc error: code = NotFound desc = could not find container \"59eec91e37756213422616f31141904928bb1589d63159cf247432d5021600e7\": container with ID starting with 59eec91e37756213422616f31141904928bb1589d63159cf247432d5021600e7 not found: ID does not exist" Jul 03 10:08:10.480537 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.480518 2573 scope.go:117] "RemoveContainer" containerID="47fca4da61224c44d2aec55d3dc9003d5ac0fff33f0a6232cee720c76001930a" Jul 03 10:08:10.480774 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:08:10.480753 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47fca4da61224c44d2aec55d3dc9003d5ac0fff33f0a6232cee720c76001930a\": container with ID starting with 47fca4da61224c44d2aec55d3dc9003d5ac0fff33f0a6232cee720c76001930a not found: ID does not exist" containerID="47fca4da61224c44d2aec55d3dc9003d5ac0fff33f0a6232cee720c76001930a" Jul 03 10:08:10.480888 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.480777 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47fca4da61224c44d2aec55d3dc9003d5ac0fff33f0a6232cee720c76001930a"} err="failed to get container status \"47fca4da61224c44d2aec55d3dc9003d5ac0fff33f0a6232cee720c76001930a\": rpc error: code = NotFound desc = could not find container \"47fca4da61224c44d2aec55d3dc9003d5ac0fff33f0a6232cee720c76001930a\": container with ID starting with 47fca4da61224c44d2aec55d3dc9003d5ac0fff33f0a6232cee720c76001930a not found: ID does not exist" Jul 03 10:08:10.480888 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.480791 2573 scope.go:117] "RemoveContainer" containerID="55385124e578d5e7164e179d51e2ffde7e82d3d374fe0c88c5d95fa3adfc6d7c" Jul 03 10:08:10.481021 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:08:10.481006 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55385124e578d5e7164e179d51e2ffde7e82d3d374fe0c88c5d95fa3adfc6d7c\": container with ID starting with 55385124e578d5e7164e179d51e2ffde7e82d3d374fe0c88c5d95fa3adfc6d7c not found: ID does not exist" containerID="55385124e578d5e7164e179d51e2ffde7e82d3d374fe0c88c5d95fa3adfc6d7c" Jul 03 10:08:10.481065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.481023 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55385124e578d5e7164e179d51e2ffde7e82d3d374fe0c88c5d95fa3adfc6d7c"} err="failed to get container status \"55385124e578d5e7164e179d51e2ffde7e82d3d374fe0c88c5d95fa3adfc6d7c\": rpc error: code = NotFound desc = could not find container \"55385124e578d5e7164e179d51e2ffde7e82d3d374fe0c88c5d95fa3adfc6d7c\": container with ID starting with 55385124e578d5e7164e179d51e2ffde7e82d3d374fe0c88c5d95fa3adfc6d7c not found: ID does not exist" Jul 03 10:08:10.481065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.481034 2573 scope.go:117] "RemoveContainer" containerID="e98bc6ee39b35c913aec207246fffbeb9b61d36851bce552490fcd73d7efc301" Jul 03 10:08:10.481246 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:08:10.481231 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e98bc6ee39b35c913aec207246fffbeb9b61d36851bce552490fcd73d7efc301\": container with ID starting with e98bc6ee39b35c913aec207246fffbeb9b61d36851bce552490fcd73d7efc301 not found: ID does not exist" containerID="e98bc6ee39b35c913aec207246fffbeb9b61d36851bce552490fcd73d7efc301" Jul 03 10:08:10.481284 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.481249 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e98bc6ee39b35c913aec207246fffbeb9b61d36851bce552490fcd73d7efc301"} err="failed to get container status \"e98bc6ee39b35c913aec207246fffbeb9b61d36851bce552490fcd73d7efc301\": rpc error: code = NotFound desc = could not find container \"e98bc6ee39b35c913aec207246fffbeb9b61d36851bce552490fcd73d7efc301\": container with ID starting with e98bc6ee39b35c913aec207246fffbeb9b61d36851bce552490fcd73d7efc301 not found: ID does not exist" Jul 03 10:08:10.496890 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.496857 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 03 10:08:10.497183 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497170 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="prom-label-proxy" Jul 03 10:08:10.497239 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497184 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="prom-label-proxy" Jul 03 10:08:10.497239 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497199 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="alertmanager" Jul 03 10:08:10.497239 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497204 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="alertmanager" Jul 03 10:08:10.497239 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497212 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="kube-rbac-proxy" Jul 03 10:08:10.497239 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497218 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="kube-rbac-proxy" Jul 03 10:08:10.497239 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497225 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="config-reloader" Jul 03 10:08:10.497239 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497230 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="config-reloader" Jul 03 10:08:10.497239 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497237 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="kube-rbac-proxy-web" Jul 03 10:08:10.497239 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497243 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="kube-rbac-proxy-web" Jul 03 10:08:10.497521 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497253 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="kube-rbac-proxy-metric" Jul 03 10:08:10.497521 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497258 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="kube-rbac-proxy-metric" Jul 03 10:08:10.497521 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497266 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="init-config-reloader" Jul 03 10:08:10.497521 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497271 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="init-config-reloader" Jul 03 10:08:10.497521 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497279 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86c2c605-065a-454b-bcb2-5cc057993daf" containerName="console" Jul 03 10:08:10.497521 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497284 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="86c2c605-065a-454b-bcb2-5cc057993daf" containerName="console" Jul 03 10:08:10.497521 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497328 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="kube-rbac-proxy-metric" Jul 03 10:08:10.497521 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497336 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="kube-rbac-proxy" Jul 03 10:08:10.497521 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497343 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="prom-label-proxy" Jul 03 10:08:10.497521 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497349 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="kube-rbac-proxy-web" Jul 03 10:08:10.497521 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497354 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="86c2c605-065a-454b-bcb2-5cc057993daf" containerName="console" Jul 03 10:08:10.497521 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497361 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="config-reloader" Jul 03 10:08:10.497521 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.497366 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" containerName="alertmanager" Jul 03 10:08:10.501282 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.501264 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.504751 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.504707 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 03 10:08:10.504857 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.504753 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 03 10:08:10.505016 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.505004 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 03 10:08:10.505051 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.505030 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 03 10:08:10.505723 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.505706 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 03 10:08:10.505852 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.505820 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 03 10:08:10.505927 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.505911 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 03 10:08:10.506126 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.506111 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 03 10:08:10.506481 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.506462 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-hp98j\"" Jul 03 10:08:10.511665 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.511645 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 03 10:08:10.518995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.518975 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 03 10:08:10.546629 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.546599 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.546824 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.546645 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.546824 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.546671 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c3dca71e-192e-4cf2-932b-a8e878872f70-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.546824 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.546772 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-config-volume\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.546824 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.546798 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.546824 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.546819 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.547018 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.546847 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c3dca71e-192e-4cf2-932b-a8e878872f70-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.547018 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.546876 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.547018 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.546901 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-web-config\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.547018 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.546956 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c3dca71e-192e-4cf2-932b-a8e878872f70-config-out\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.547018 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.546999 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3dca71e-192e-4cf2-932b-a8e878872f70-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.547223 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.547068 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kks6v\" (UniqueName: \"kubernetes.io/projected/c3dca71e-192e-4cf2-932b-a8e878872f70-kube-api-access-kks6v\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.547223 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.547119 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3dca71e-192e-4cf2-932b-a8e878872f70-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.562200 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.562173 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4615fd38-b169-4b84-90f0-ca1ab940acd5" path="/var/lib/kubelet/pods/4615fd38-b169-4b84-90f0-ca1ab940acd5/volumes" Jul 03 10:08:10.648331 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.648293 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-web-config\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.648485 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.648377 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c3dca71e-192e-4cf2-932b-a8e878872f70-config-out\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.648485 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.648453 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3dca71e-192e-4cf2-932b-a8e878872f70-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.648851 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.648816 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kks6v\" (UniqueName: \"kubernetes.io/projected/c3dca71e-192e-4cf2-932b-a8e878872f70-kube-api-access-kks6v\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.648968 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.648860 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3dca71e-192e-4cf2-932b-a8e878872f70-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.648968 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.648887 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.648968 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.648937 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.649141 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.649125 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c3dca71e-192e-4cf2-932b-a8e878872f70-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.649199 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.649177 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-config-volume\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.649253 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.649202 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.649253 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.649232 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.649357 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.649273 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c3dca71e-192e-4cf2-932b-a8e878872f70-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.649357 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.649307 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.650081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.649551 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3dca71e-192e-4cf2-932b-a8e878872f70-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.650081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.649565 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3dca71e-192e-4cf2-932b-a8e878872f70-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.650081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.649760 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c3dca71e-192e-4cf2-932b-a8e878872f70-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.651406 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.651381 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c3dca71e-192e-4cf2-932b-a8e878872f70-config-out\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.651501 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.651442 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-web-config\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.652051 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.652029 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c3dca71e-192e-4cf2-932b-a8e878872f70-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.652051 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.652040 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.652493 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.652474 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-config-volume\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.652810 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.652791 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.652873 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.652850 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.652964 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.652946 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.653198 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.653184 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c3dca71e-192e-4cf2-932b-a8e878872f70-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.663115 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.663096 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kks6v\" (UniqueName: \"kubernetes.io/projected/c3dca71e-192e-4cf2-932b-a8e878872f70-kube-api-access-kks6v\") pod \"alertmanager-main-0\" (UID: \"c3dca71e-192e-4cf2-932b-a8e878872f70\") " pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.811294 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.811212 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 03 10:08:10.949961 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:10.949936 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 03 10:08:10.952484 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:08:10.952457 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3dca71e_192e_4cf2_932b_a8e878872f70.slice/crio-d96c604485853bf50f650aa21da7f4b2cfec353f47ab1be854b16aeb51d8c45b WatchSource:0}: Error finding container d96c604485853bf50f650aa21da7f4b2cfec353f47ab1be854b16aeb51d8c45b: Status 404 returned error can't find the container with id d96c604485853bf50f650aa21da7f4b2cfec353f47ab1be854b16aeb51d8c45b Jul 03 10:08:11.426486 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:11.426451 2573 generic.go:356] "Generic (PLEG): container finished" podID="c3dca71e-192e-4cf2-932b-a8e878872f70" containerID="4ffe7468f5917a263ea0a69b14cc3c640dac4d857371cacce9cdcee59e5574e7" exitCode=0 Jul 03 10:08:11.426885 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:11.426499 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3dca71e-192e-4cf2-932b-a8e878872f70","Type":"ContainerDied","Data":"4ffe7468f5917a263ea0a69b14cc3c640dac4d857371cacce9cdcee59e5574e7"} Jul 03 10:08:11.426885 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:11.426567 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3dca71e-192e-4cf2-932b-a8e878872f70","Type":"ContainerStarted","Data":"d96c604485853bf50f650aa21da7f4b2cfec353f47ab1be854b16aeb51d8c45b"} Jul 03 10:08:12.328196 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.328165 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-754fffd6f7-2n7px"] Jul 03 10:08:12.330914 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.330898 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.333686 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.333664 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-f6dfv\"" Jul 03 10:08:12.333838 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.333728 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 03 10:08:12.333909 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.333858 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 03 10:08:12.334179 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.334163 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 03 10:08:12.334246 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.334186 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 03 10:08:12.334246 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.334214 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 03 10:08:12.339129 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.339113 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 03 10:08:12.345007 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.344987 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-754fffd6f7-2n7px"] Jul 03 10:08:12.434442 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.434405 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3dca71e-192e-4cf2-932b-a8e878872f70","Type":"ContainerStarted","Data":"682f3a92408383b9c9df8542e2cd05b5cb578bdd9da8be9584014d27b045bb72"} Jul 03 10:08:12.434825 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.434468 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3dca71e-192e-4cf2-932b-a8e878872f70","Type":"ContainerStarted","Data":"97f41b953f152cd5bc9a0d8abf41f5e92811644353cd1ee82471f1949bfe267e"} Jul 03 10:08:12.434825 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.434495 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3dca71e-192e-4cf2-932b-a8e878872f70","Type":"ContainerStarted","Data":"af169bca4de01d3b846c543c820e18ff81b7000b28df44d8186f5df484ff2d80"} Jul 03 10:08:12.434825 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.434519 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3dca71e-192e-4cf2-932b-a8e878872f70","Type":"ContainerStarted","Data":"5d1a2264b03b492c9a748fb09950ed1ca64bff01e1685d9f290502087154af08"} Jul 03 10:08:12.434825 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.434543 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3dca71e-192e-4cf2-932b-a8e878872f70","Type":"ContainerStarted","Data":"73be9473e8cc9eefdfb05312f0802bb2f9a604b825483314b5303f279351b0b2"} Jul 03 10:08:12.434825 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.434567 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c3dca71e-192e-4cf2-932b-a8e878872f70","Type":"ContainerStarted","Data":"ba70e7c8cd3fb89af028cb1dfbf9a4215427835ec00e844fdb0caa348adcb4ab"} Jul 03 10:08:12.462215 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.462156 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.443863203 podStartE2EDuration="2.462138398s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.275195ms" isStatefulPod=false podCreationTimestamp="2026-07-03 10:08:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:08:12.460890204 +0000 UTC m=+147.372779508" watchObservedRunningTime="2026-07-03 10:08:12.462138398 +0000 UTC m=+147.374027722" Jul 03 10:08:12.466400 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.466377 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/92777b98-f96f-4bb5-927f-5a4b46356a0c-secret-telemeter-client\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.466478 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.466412 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/92777b98-f96f-4bb5-927f-5a4b46356a0c-federate-client-tls\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.466478 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.466433 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92777b98-f96f-4bb5-927f-5a4b46356a0c-metrics-client-ca\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.466478 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.466465 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/92777b98-f96f-4bb5-927f-5a4b46356a0c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.466573 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.466520 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92777b98-f96f-4bb5-927f-5a4b46356a0c-serving-certs-ca-bundle\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.466573 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.466548 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/92777b98-f96f-4bb5-927f-5a4b46356a0c-telemeter-client-tls\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.466634 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.466582 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92777b98-f96f-4bb5-927f-5a4b46356a0c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.466634 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.466621 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-454sh\" (UniqueName: \"kubernetes.io/projected/92777b98-f96f-4bb5-927f-5a4b46356a0c-kube-api-access-454sh\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.567071 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.567039 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/92777b98-f96f-4bb5-927f-5a4b46356a0c-secret-telemeter-client\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.567071 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.567072 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/92777b98-f96f-4bb5-927f-5a4b46356a0c-federate-client-tls\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.567302 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.567214 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92777b98-f96f-4bb5-927f-5a4b46356a0c-metrics-client-ca\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.567302 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.567266 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/92777b98-f96f-4bb5-927f-5a4b46356a0c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.567414 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.567372 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92777b98-f96f-4bb5-927f-5a4b46356a0c-serving-certs-ca-bundle\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.567466 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.567423 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/92777b98-f96f-4bb5-927f-5a4b46356a0c-telemeter-client-tls\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.567577 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.567562 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92777b98-f96f-4bb5-927f-5a4b46356a0c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.567630 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.567592 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-454sh\" (UniqueName: \"kubernetes.io/projected/92777b98-f96f-4bb5-927f-5a4b46356a0c-kube-api-access-454sh\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.568057 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.568025 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92777b98-f96f-4bb5-927f-5a4b46356a0c-metrics-client-ca\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.568562 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.568532 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92777b98-f96f-4bb5-927f-5a4b46356a0c-serving-certs-ca-bundle\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.568844 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.568818 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92777b98-f96f-4bb5-927f-5a4b46356a0c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.569973 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.569954 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/92777b98-f96f-4bb5-927f-5a4b46356a0c-federate-client-tls\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.570054 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.570023 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/92777b98-f96f-4bb5-927f-5a4b46356a0c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.570054 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.570031 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/92777b98-f96f-4bb5-927f-5a4b46356a0c-secret-telemeter-client\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.570263 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.570245 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/92777b98-f96f-4bb5-927f-5a4b46356a0c-telemeter-client-tls\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.575340 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.575317 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-454sh\" (UniqueName: \"kubernetes.io/projected/92777b98-f96f-4bb5-927f-5a4b46356a0c-kube-api-access-454sh\") pod \"telemeter-client-754fffd6f7-2n7px\" (UID: \"92777b98-f96f-4bb5-927f-5a4b46356a0c\") " pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.630030 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.629948 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 03 10:08:12.630285 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.630239 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="prometheus" containerID="cri-o://a0f6f8052fb1b3ea367b51ff7eddb4dcbecb0cc48ade08978305ff785a65f263" gracePeriod=600 Jul 03 10:08:12.630285 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.630256 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="thanos-sidecar" containerID="cri-o://568ba03def1e9590d1e39fda5b1dafc85c2146aaafcd946e6b29a051f5048736" gracePeriod=600 Jul 03 10:08:12.630285 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.630275 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="kube-rbac-proxy-web" containerID="cri-o://129b775dda6ab6865eb8e22d49468371b16c90e621bdb153f9f5bf17b13b7150" gracePeriod=600 Jul 03 10:08:12.630504 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.630253 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="kube-rbac-proxy" containerID="cri-o://2cfa98fa7d32e377f93c19770c62cedc6312e76c59e422d938ec18b0e5015c67" gracePeriod=600 Jul 03 10:08:12.630504 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.630314 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="config-reloader" containerID="cri-o://2c1f72b88a783a62fcd60dd0bbdcb528b54cddb1937e05cb1e43375bdc2ddc4c" gracePeriod=600 Jul 03 10:08:12.630504 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.630282 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="kube-rbac-proxy-thanos" containerID="cri-o://4faf9001514fd18bdf6596ce0cb7e031203808206aecc20e0a625dac4dee090a" gracePeriod=600 Jul 03 10:08:12.641163 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.641135 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" Jul 03 10:08:12.782066 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:12.782029 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-754fffd6f7-2n7px"] Jul 03 10:08:12.784129 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:08:12.784096 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92777b98_f96f_4bb5_927f_5a4b46356a0c.slice/crio-036cfff49c1253706d5375c227f175831edce16d05baf8b531c4898b7213685a WatchSource:0}: Error finding container 036cfff49c1253706d5375c227f175831edce16d05baf8b531c4898b7213685a: Status 404 returned error can't find the container with id 036cfff49c1253706d5375c227f175831edce16d05baf8b531c4898b7213685a Jul 03 10:08:13.444601 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.444562 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" event={"ID":"92777b98-f96f-4bb5-927f-5a4b46356a0c","Type":"ContainerStarted","Data":"036cfff49c1253706d5375c227f175831edce16d05baf8b531c4898b7213685a"} Jul 03 10:08:13.448288 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.448043 2573 generic.go:356] "Generic (PLEG): container finished" podID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerID="4faf9001514fd18bdf6596ce0cb7e031203808206aecc20e0a625dac4dee090a" exitCode=0 Jul 03 10:08:13.448288 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.448066 2573 generic.go:356] "Generic (PLEG): container finished" podID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerID="2cfa98fa7d32e377f93c19770c62cedc6312e76c59e422d938ec18b0e5015c67" exitCode=0 Jul 03 10:08:13.448288 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.448075 2573 generic.go:356] "Generic (PLEG): container finished" podID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerID="568ba03def1e9590d1e39fda5b1dafc85c2146aaafcd946e6b29a051f5048736" exitCode=0 Jul 03 10:08:13.448288 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.448083 2573 generic.go:356] "Generic (PLEG): container finished" podID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerID="2c1f72b88a783a62fcd60dd0bbdcb528b54cddb1937e05cb1e43375bdc2ddc4c" exitCode=0 Jul 03 10:08:13.448288 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.448092 2573 generic.go:356] "Generic (PLEG): container finished" podID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerID="a0f6f8052fb1b3ea367b51ff7eddb4dcbecb0cc48ade08978305ff785a65f263" exitCode=0 Jul 03 10:08:13.448288 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.448117 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af65bbd0-8f4a-48a1-82e5-97790033560b","Type":"ContainerDied","Data":"4faf9001514fd18bdf6596ce0cb7e031203808206aecc20e0a625dac4dee090a"} Jul 03 10:08:13.448288 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.448195 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af65bbd0-8f4a-48a1-82e5-97790033560b","Type":"ContainerDied","Data":"2cfa98fa7d32e377f93c19770c62cedc6312e76c59e422d938ec18b0e5015c67"} Jul 03 10:08:13.448288 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.448236 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af65bbd0-8f4a-48a1-82e5-97790033560b","Type":"ContainerDied","Data":"568ba03def1e9590d1e39fda5b1dafc85c2146aaafcd946e6b29a051f5048736"} Jul 03 10:08:13.448288 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.448280 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af65bbd0-8f4a-48a1-82e5-97790033560b","Type":"ContainerDied","Data":"2c1f72b88a783a62fcd60dd0bbdcb528b54cddb1937e05cb1e43375bdc2ddc4c"} Jul 03 10:08:13.448706 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.448318 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af65bbd0-8f4a-48a1-82e5-97790033560b","Type":"ContainerDied","Data":"a0f6f8052fb1b3ea367b51ff7eddb4dcbecb0cc48ade08978305ff785a65f263"} Jul 03 10:08:13.886220 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.886194 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:13.979899 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.979823 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-db\") pod \"af65bbd0-8f4a-48a1-82e5-97790033560b\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " Jul 03 10:08:13.979899 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.979873 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-trusted-ca-bundle\") pod \"af65bbd0-8f4a-48a1-82e5-97790033560b\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " Jul 03 10:08:13.979899 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.979892 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af65bbd0-8f4a-48a1-82e5-97790033560b-tls-assets\") pod \"af65bbd0-8f4a-48a1-82e5-97790033560b\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " Jul 03 10:08:13.980164 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.979928 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af65bbd0-8f4a-48a1-82e5-97790033560b-config-out\") pod \"af65bbd0-8f4a-48a1-82e5-97790033560b\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " Jul 03 10:08:13.980164 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.979946 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-configmap-serving-certs-ca-bundle\") pod \"af65bbd0-8f4a-48a1-82e5-97790033560b\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " Jul 03 10:08:13.980164 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.979966 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-rulefiles-0\") pod \"af65bbd0-8f4a-48a1-82e5-97790033560b\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " Jul 03 10:08:13.980164 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.979996 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-prometheus-k8s-tls\") pod \"af65bbd0-8f4a-48a1-82e5-97790033560b\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " Jul 03 10:08:13.980164 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.980029 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"af65bbd0-8f4a-48a1-82e5-97790033560b\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " Jul 03 10:08:13.980164 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.980070 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rp95r\" (UniqueName: \"kubernetes.io/projected/af65bbd0-8f4a-48a1-82e5-97790033560b-kube-api-access-rp95r\") pod \"af65bbd0-8f4a-48a1-82e5-97790033560b\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " Jul 03 10:08:13.980164 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.980097 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-configmap-kubelet-serving-ca-bundle\") pod \"af65bbd0-8f4a-48a1-82e5-97790033560b\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " Jul 03 10:08:13.980164 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.980132 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"af65bbd0-8f4a-48a1-82e5-97790033560b\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " Jul 03 10:08:13.980164 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.980157 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-metrics-client-certs\") pod \"af65bbd0-8f4a-48a1-82e5-97790033560b\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " Jul 03 10:08:13.980598 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.980189 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-web-config\") pod \"af65bbd0-8f4a-48a1-82e5-97790033560b\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " Jul 03 10:08:13.980598 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.980217 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-rulefiles-1\") pod \"af65bbd0-8f4a-48a1-82e5-97790033560b\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " Jul 03 10:08:13.980598 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.980243 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-configmap-metrics-client-ca\") pod \"af65bbd0-8f4a-48a1-82e5-97790033560b\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " Jul 03 10:08:13.980598 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.980267 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-rulefiles-2\") pod \"af65bbd0-8f4a-48a1-82e5-97790033560b\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " Jul 03 10:08:13.980598 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.980291 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-config\") pod \"af65bbd0-8f4a-48a1-82e5-97790033560b\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " Jul 03 10:08:13.980598 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.980329 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-kube-rbac-proxy\") pod \"af65bbd0-8f4a-48a1-82e5-97790033560b\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " Jul 03 10:08:13.980598 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.980264 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "af65bbd0-8f4a-48a1-82e5-97790033560b" (UID: "af65bbd0-8f4a-48a1-82e5-97790033560b"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:08:13.980598 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.980369 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-grpc-tls\") pod \"af65bbd0-8f4a-48a1-82e5-97790033560b\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " Jul 03 10:08:13.980598 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.980425 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-thanos-prometheus-http-client-file\") pod \"af65bbd0-8f4a-48a1-82e5-97790033560b\" (UID: \"af65bbd0-8f4a-48a1-82e5-97790033560b\") " Jul 03 10:08:13.980598 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.980456 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "af65bbd0-8f4a-48a1-82e5-97790033560b" (UID: "af65bbd0-8f4a-48a1-82e5-97790033560b"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:08:13.981128 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.981040 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "af65bbd0-8f4a-48a1-82e5-97790033560b" (UID: "af65bbd0-8f4a-48a1-82e5-97790033560b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:08:13.981216 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.981173 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "af65bbd0-8f4a-48a1-82e5-97790033560b" (UID: "af65bbd0-8f4a-48a1-82e5-97790033560b"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:08:13.981786 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.981755 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "af65bbd0-8f4a-48a1-82e5-97790033560b" (UID: "af65bbd0-8f4a-48a1-82e5-97790033560b"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:08:13.981922 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.981828 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "af65bbd0-8f4a-48a1-82e5-97790033560b" (UID: "af65bbd0-8f4a-48a1-82e5-97790033560b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:08:13.982485 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.982234 2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:13.982485 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.982296 2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-db\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:13.982485 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.982315 2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-trusted-ca-bundle\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:13.982485 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.982334 2573 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:13.982485 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.982231 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "af65bbd0-8f4a-48a1-82e5-97790033560b" (UID: "af65bbd0-8f4a-48a1-82e5-97790033560b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:08:13.982485 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.982350 2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:13.982485 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.982376 2573 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:13.983190 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.983160 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "af65bbd0-8f4a-48a1-82e5-97790033560b" (UID: "af65bbd0-8f4a-48a1-82e5-97790033560b"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:08:13.983827 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.983799 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "af65bbd0-8f4a-48a1-82e5-97790033560b" (UID: "af65bbd0-8f4a-48a1-82e5-97790033560b"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:08:13.985043 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.985012 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af65bbd0-8f4a-48a1-82e5-97790033560b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "af65bbd0-8f4a-48a1-82e5-97790033560b" (UID: "af65bbd0-8f4a-48a1-82e5-97790033560b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:08:13.985137 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.985073 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-config" (OuterVolumeSpecName: "config") pod "af65bbd0-8f4a-48a1-82e5-97790033560b" (UID: "af65bbd0-8f4a-48a1-82e5-97790033560b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:08:13.985305 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.985277 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af65bbd0-8f4a-48a1-82e5-97790033560b-config-out" (OuterVolumeSpecName: "config-out") pod "af65bbd0-8f4a-48a1-82e5-97790033560b" (UID: "af65bbd0-8f4a-48a1-82e5-97790033560b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:08:13.985305 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.985289 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "af65bbd0-8f4a-48a1-82e5-97790033560b" (UID: "af65bbd0-8f4a-48a1-82e5-97790033560b"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:08:13.985496 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.985463 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af65bbd0-8f4a-48a1-82e5-97790033560b-kube-api-access-rp95r" (OuterVolumeSpecName: "kube-api-access-rp95r") pod "af65bbd0-8f4a-48a1-82e5-97790033560b" (UID: "af65bbd0-8f4a-48a1-82e5-97790033560b"). InnerVolumeSpecName "kube-api-access-rp95r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:08:13.985496 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.985471 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "af65bbd0-8f4a-48a1-82e5-97790033560b" (UID: "af65bbd0-8f4a-48a1-82e5-97790033560b"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:08:13.985611 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.985547 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "af65bbd0-8f4a-48a1-82e5-97790033560b" (UID: "af65bbd0-8f4a-48a1-82e5-97790033560b"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:08:13.985691 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.985666 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "af65bbd0-8f4a-48a1-82e5-97790033560b" (UID: "af65bbd0-8f4a-48a1-82e5-97790033560b"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:08:13.985783 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.985717 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "af65bbd0-8f4a-48a1-82e5-97790033560b" (UID: "af65bbd0-8f4a-48a1-82e5-97790033560b"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:08:13.985899 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.985877 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "af65bbd0-8f4a-48a1-82e5-97790033560b" (UID: "af65bbd0-8f4a-48a1-82e5-97790033560b"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:08:13.995441 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:13.995418 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-web-config" (OuterVolumeSpecName: "web-config") pod "af65bbd0-8f4a-48a1-82e5-97790033560b" (UID: "af65bbd0-8f4a-48a1-82e5-97790033560b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:08:14.083526 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.083491 2573 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-kube-rbac-proxy\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:14.083526 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.083528 2573 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-grpc-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:14.083779 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.083546 2573 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-thanos-prometheus-http-client-file\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:14.083779 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.083562 2573 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af65bbd0-8f4a-48a1-82e5-97790033560b-tls-assets\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:14.083779 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.083575 2573 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af65bbd0-8f4a-48a1-82e5-97790033560b-config-out\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:14.083779 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.083587 2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-prometheus-k8s-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:14.083779 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.083603 2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:14.083779 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.083616 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rp95r\" (UniqueName: \"kubernetes.io/projected/af65bbd0-8f4a-48a1-82e5-97790033560b-kube-api-access-rp95r\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:14.083779 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.083626 2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:14.083779 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.083640 2573 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-secret-metrics-client-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:14.083779 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.083650 2573 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-web-config\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:14.083779 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.083658 2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:14.083779 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.083668 2573 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af65bbd0-8f4a-48a1-82e5-97790033560b-configmap-metrics-client-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:14.083779 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.083676 2573 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/af65bbd0-8f4a-48a1-82e5-97790033560b-config\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:14.455318 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.455287 2573 generic.go:356] "Generic (PLEG): container finished" podID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerID="129b775dda6ab6865eb8e22d49468371b16c90e621bdb153f9f5bf17b13b7150" exitCode=0 Jul 03 10:08:14.455692 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.455359 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.455692 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.455360 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af65bbd0-8f4a-48a1-82e5-97790033560b","Type":"ContainerDied","Data":"129b775dda6ab6865eb8e22d49468371b16c90e621bdb153f9f5bf17b13b7150"} Jul 03 10:08:14.455692 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.455488 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af65bbd0-8f4a-48a1-82e5-97790033560b","Type":"ContainerDied","Data":"0b36b20a856613e04c88ccecac7499ad6e9763c285abe9edde4f0958ffc05e43"} Jul 03 10:08:14.455692 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.455520 2573 scope.go:117] "RemoveContainer" containerID="4faf9001514fd18bdf6596ce0cb7e031203808206aecc20e0a625dac4dee090a" Jul 03 10:08:14.466516 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.466417 2573 scope.go:117] "RemoveContainer" containerID="2cfa98fa7d32e377f93c19770c62cedc6312e76c59e422d938ec18b0e5015c67" Jul 03 10:08:14.475264 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.475238 2573 scope.go:117] "RemoveContainer" containerID="129b775dda6ab6865eb8e22d49468371b16c90e621bdb153f9f5bf17b13b7150" Jul 03 10:08:14.482697 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.482674 2573 scope.go:117] "RemoveContainer" containerID="568ba03def1e9590d1e39fda5b1dafc85c2146aaafcd946e6b29a051f5048736" Jul 03 10:08:14.490351 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.490331 2573 scope.go:117] "RemoveContainer" containerID="2c1f72b88a783a62fcd60dd0bbdcb528b54cddb1937e05cb1e43375bdc2ddc4c" Jul 03 10:08:14.498146 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.498122 2573 scope.go:117] "RemoveContainer" containerID="a0f6f8052fb1b3ea367b51ff7eddb4dcbecb0cc48ade08978305ff785a65f263" Jul 03 10:08:14.504121 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.504095 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 03 10:08:14.508000 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.507969 2573 scope.go:117] "RemoveContainer" containerID="b5276ec3e3ce9a937da4598ddc3877e67d75cd757c875a65d8b4d39a062c49a4" Jul 03 10:08:14.513371 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.513351 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 03 10:08:14.517324 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.517300 2573 scope.go:117] "RemoveContainer" containerID="4faf9001514fd18bdf6596ce0cb7e031203808206aecc20e0a625dac4dee090a" Jul 03 10:08:14.517665 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:08:14.517641 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4faf9001514fd18bdf6596ce0cb7e031203808206aecc20e0a625dac4dee090a\": container with ID starting with 4faf9001514fd18bdf6596ce0cb7e031203808206aecc20e0a625dac4dee090a not found: ID does not exist" containerID="4faf9001514fd18bdf6596ce0cb7e031203808206aecc20e0a625dac4dee090a" Jul 03 10:08:14.517778 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.517679 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4faf9001514fd18bdf6596ce0cb7e031203808206aecc20e0a625dac4dee090a"} err="failed to get container status \"4faf9001514fd18bdf6596ce0cb7e031203808206aecc20e0a625dac4dee090a\": rpc error: code = NotFound desc = could not find container \"4faf9001514fd18bdf6596ce0cb7e031203808206aecc20e0a625dac4dee090a\": container with ID starting with 4faf9001514fd18bdf6596ce0cb7e031203808206aecc20e0a625dac4dee090a not found: ID does not exist" Jul 03 10:08:14.517778 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.517707 2573 scope.go:117] "RemoveContainer" containerID="2cfa98fa7d32e377f93c19770c62cedc6312e76c59e422d938ec18b0e5015c67" Jul 03 10:08:14.518029 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:08:14.518010 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cfa98fa7d32e377f93c19770c62cedc6312e76c59e422d938ec18b0e5015c67\": container with ID starting with 2cfa98fa7d32e377f93c19770c62cedc6312e76c59e422d938ec18b0e5015c67 not found: ID does not exist" containerID="2cfa98fa7d32e377f93c19770c62cedc6312e76c59e422d938ec18b0e5015c67" Jul 03 10:08:14.518115 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.518038 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cfa98fa7d32e377f93c19770c62cedc6312e76c59e422d938ec18b0e5015c67"} err="failed to get container status \"2cfa98fa7d32e377f93c19770c62cedc6312e76c59e422d938ec18b0e5015c67\": rpc error: code = NotFound desc = could not find container \"2cfa98fa7d32e377f93c19770c62cedc6312e76c59e422d938ec18b0e5015c67\": container with ID starting with 2cfa98fa7d32e377f93c19770c62cedc6312e76c59e422d938ec18b0e5015c67 not found: ID does not exist" Jul 03 10:08:14.518115 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.518060 2573 scope.go:117] "RemoveContainer" containerID="129b775dda6ab6865eb8e22d49468371b16c90e621bdb153f9f5bf17b13b7150" Jul 03 10:08:14.518347 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:08:14.518328 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"129b775dda6ab6865eb8e22d49468371b16c90e621bdb153f9f5bf17b13b7150\": container with ID starting with 129b775dda6ab6865eb8e22d49468371b16c90e621bdb153f9f5bf17b13b7150 not found: ID does not exist" containerID="129b775dda6ab6865eb8e22d49468371b16c90e621bdb153f9f5bf17b13b7150" Jul 03 10:08:14.518404 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.518351 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"129b775dda6ab6865eb8e22d49468371b16c90e621bdb153f9f5bf17b13b7150"} err="failed to get container status \"129b775dda6ab6865eb8e22d49468371b16c90e621bdb153f9f5bf17b13b7150\": rpc error: code = NotFound desc = could not find container \"129b775dda6ab6865eb8e22d49468371b16c90e621bdb153f9f5bf17b13b7150\": container with ID starting with 129b775dda6ab6865eb8e22d49468371b16c90e621bdb153f9f5bf17b13b7150 not found: ID does not exist" Jul 03 10:08:14.518404 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.518367 2573 scope.go:117] "RemoveContainer" containerID="568ba03def1e9590d1e39fda5b1dafc85c2146aaafcd946e6b29a051f5048736" Jul 03 10:08:14.518618 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:08:14.518596 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"568ba03def1e9590d1e39fda5b1dafc85c2146aaafcd946e6b29a051f5048736\": container with ID starting with 568ba03def1e9590d1e39fda5b1dafc85c2146aaafcd946e6b29a051f5048736 not found: ID does not exist" containerID="568ba03def1e9590d1e39fda5b1dafc85c2146aaafcd946e6b29a051f5048736" Jul 03 10:08:14.518660 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.518624 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"568ba03def1e9590d1e39fda5b1dafc85c2146aaafcd946e6b29a051f5048736"} err="failed to get container status \"568ba03def1e9590d1e39fda5b1dafc85c2146aaafcd946e6b29a051f5048736\": rpc error: code = NotFound desc = could not find container \"568ba03def1e9590d1e39fda5b1dafc85c2146aaafcd946e6b29a051f5048736\": container with ID starting with 568ba03def1e9590d1e39fda5b1dafc85c2146aaafcd946e6b29a051f5048736 not found: ID does not exist" Jul 03 10:08:14.518660 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.518639 2573 scope.go:117] "RemoveContainer" containerID="2c1f72b88a783a62fcd60dd0bbdcb528b54cddb1937e05cb1e43375bdc2ddc4c" Jul 03 10:08:14.518885 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:08:14.518860 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c1f72b88a783a62fcd60dd0bbdcb528b54cddb1937e05cb1e43375bdc2ddc4c\": container with ID starting with 2c1f72b88a783a62fcd60dd0bbdcb528b54cddb1937e05cb1e43375bdc2ddc4c not found: ID does not exist" containerID="2c1f72b88a783a62fcd60dd0bbdcb528b54cddb1937e05cb1e43375bdc2ddc4c" Jul 03 10:08:14.518992 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.518888 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c1f72b88a783a62fcd60dd0bbdcb528b54cddb1937e05cb1e43375bdc2ddc4c"} err="failed to get container status \"2c1f72b88a783a62fcd60dd0bbdcb528b54cddb1937e05cb1e43375bdc2ddc4c\": rpc error: code = NotFound desc = could not find container \"2c1f72b88a783a62fcd60dd0bbdcb528b54cddb1937e05cb1e43375bdc2ddc4c\": container with ID starting with 2c1f72b88a783a62fcd60dd0bbdcb528b54cddb1937e05cb1e43375bdc2ddc4c not found: ID does not exist" Jul 03 10:08:14.518992 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.518909 2573 scope.go:117] "RemoveContainer" containerID="a0f6f8052fb1b3ea367b51ff7eddb4dcbecb0cc48ade08978305ff785a65f263" Jul 03 10:08:14.519182 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:08:14.519161 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0f6f8052fb1b3ea367b51ff7eddb4dcbecb0cc48ade08978305ff785a65f263\": container with ID starting with a0f6f8052fb1b3ea367b51ff7eddb4dcbecb0cc48ade08978305ff785a65f263 not found: ID does not exist" containerID="a0f6f8052fb1b3ea367b51ff7eddb4dcbecb0cc48ade08978305ff785a65f263" Jul 03 10:08:14.519237 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.519187 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0f6f8052fb1b3ea367b51ff7eddb4dcbecb0cc48ade08978305ff785a65f263"} err="failed to get container status \"a0f6f8052fb1b3ea367b51ff7eddb4dcbecb0cc48ade08978305ff785a65f263\": rpc error: code = NotFound desc = could not find container \"a0f6f8052fb1b3ea367b51ff7eddb4dcbecb0cc48ade08978305ff785a65f263\": container with ID starting with a0f6f8052fb1b3ea367b51ff7eddb4dcbecb0cc48ade08978305ff785a65f263 not found: ID does not exist" Jul 03 10:08:14.519237 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.519205 2573 scope.go:117] "RemoveContainer" containerID="b5276ec3e3ce9a937da4598ddc3877e67d75cd757c875a65d8b4d39a062c49a4" Jul 03 10:08:14.519456 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:08:14.519431 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5276ec3e3ce9a937da4598ddc3877e67d75cd757c875a65d8b4d39a062c49a4\": container with ID starting with b5276ec3e3ce9a937da4598ddc3877e67d75cd757c875a65d8b4d39a062c49a4 not found: ID does not exist" containerID="b5276ec3e3ce9a937da4598ddc3877e67d75cd757c875a65d8b4d39a062c49a4" Jul 03 10:08:14.519525 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.519461 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5276ec3e3ce9a937da4598ddc3877e67d75cd757c875a65d8b4d39a062c49a4"} err="failed to get container status \"b5276ec3e3ce9a937da4598ddc3877e67d75cd757c875a65d8b4d39a062c49a4\": rpc error: code = NotFound desc = could not find container \"b5276ec3e3ce9a937da4598ddc3877e67d75cd757c875a65d8b4d39a062c49a4\": container with ID starting with b5276ec3e3ce9a937da4598ddc3877e67d75cd757c875a65d8b4d39a062c49a4 not found: ID does not exist" Jul 03 10:08:14.555429 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.555377 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 03 10:08:14.555827 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.555810 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="init-config-reloader" Jul 03 10:08:14.555935 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.555830 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="init-config-reloader" Jul 03 10:08:14.555935 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.555844 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="kube-rbac-proxy" Jul 03 10:08:14.555935 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.555853 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="kube-rbac-proxy" Jul 03 10:08:14.555935 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.555875 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="kube-rbac-proxy-thanos" Jul 03 10:08:14.555935 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.555884 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="kube-rbac-proxy-thanos" Jul 03 10:08:14.555935 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.555896 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="prometheus" Jul 03 10:08:14.555935 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.555905 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="prometheus" Jul 03 10:08:14.555935 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.555918 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="kube-rbac-proxy-web" Jul 03 10:08:14.555935 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.555926 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="kube-rbac-proxy-web" Jul 03 10:08:14.555935 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.555937 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="config-reloader" Jul 03 10:08:14.556481 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.555947 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="config-reloader" Jul 03 10:08:14.556481 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.555960 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="thanos-sidecar" Jul 03 10:08:14.556481 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.555969 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="thanos-sidecar" Jul 03 10:08:14.556481 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.556106 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="kube-rbac-proxy-thanos" Jul 03 10:08:14.556481 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.556145 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="prometheus" Jul 03 10:08:14.556481 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.556157 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="thanos-sidecar" Jul 03 10:08:14.556481 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.556168 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="kube-rbac-proxy" Jul 03 10:08:14.556481 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.556244 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="config-reloader" Jul 03 10:08:14.556481 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.556270 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" containerName="kube-rbac-proxy-web" Jul 03 10:08:14.561120 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.561096 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.566524 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.566476 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cb1190dg0om3g\"" Jul 03 10:08:14.566651 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.566596 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-m4p7n\"" Jul 03 10:08:14.566933 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.566915 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 03 10:08:14.567346 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.567145 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 03 10:08:14.567346 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.567183 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 03 10:08:14.567878 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.567666 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 03 10:08:14.567878 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.567703 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 03 10:08:14.567878 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.567807 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 03 10:08:14.568080 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.567900 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 03 10:08:14.568080 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.567919 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 03 10:08:14.568171 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.568127 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 03 10:08:14.568654 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.568310 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 03 10:08:14.568654 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.568371 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 03 10:08:14.568654 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.568460 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af65bbd0-8f4a-48a1-82e5-97790033560b" path="/var/lib/kubelet/pods/af65bbd0-8f4a-48a1-82e5-97790033560b/volumes" Jul 03 10:08:14.568654 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.568539 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 03 10:08:14.571058 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.570974 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 03 10:08:14.575268 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.575252 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 03 10:08:14.582577 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.582554 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 03 10:08:14.688663 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.688566 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.688663 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.688626 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/47b2a6a3-c715-4a87-b570-517b831398c0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.688921 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.688679 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.688921 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.688775 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-web-config\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.688921 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.688813 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.688921 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.688840 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.688921 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.688866 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlpsn\" (UniqueName: \"kubernetes.io/projected/47b2a6a3-c715-4a87-b570-517b831398c0-kube-api-access-xlpsn\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.688921 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.688896 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/47b2a6a3-c715-4a87-b570-517b831398c0-config-out\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.688921 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.688921 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.689367 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.688946 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.689367 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.688969 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.689367 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.689001 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.689367 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.689025 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/47b2a6a3-c715-4a87-b570-517b831398c0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.689367 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.689049 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-config\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.689367 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.689079 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.689367 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.689110 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.689367 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.689247 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.689367 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.689275 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.689367 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.689330 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.689858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.689376 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.790125 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.790041 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.790125 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.790092 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.790355 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.790131 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.790355 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.790162 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/47b2a6a3-c715-4a87-b570-517b831398c0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.790355 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.790190 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.790355 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.790229 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-web-config\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.790355 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.790266 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.790355 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.790296 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.790355 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.790325 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlpsn\" (UniqueName: \"kubernetes.io/projected/47b2a6a3-c715-4a87-b570-517b831398c0-kube-api-access-xlpsn\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.790355 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.790354 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/47b2a6a3-c715-4a87-b570-517b831398c0-config-out\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.790775 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.790379 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.790775 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.790407 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.790775 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.790432 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.790775 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.790461 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.790775 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.790484 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/47b2a6a3-c715-4a87-b570-517b831398c0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.790775 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.790512 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-config\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.790775 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.790543 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.790775 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.790573 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.790775 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.790622 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.790775 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.790662 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.793602 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.792258 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.793602 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.793261 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.793602 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.793301 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.793602 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.793382 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.793892 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.793864 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/47b2a6a3-c715-4a87-b570-517b831398c0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.794536 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.794514 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.794679 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.794525 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.795795 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.795169 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.795795 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.795291 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-web-config\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.795795 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.795668 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.796178 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.796103 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/47b2a6a3-c715-4a87-b570-517b831398c0-config-out\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.796178 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.796109 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-config\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.796346 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.796207 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/47b2a6a3-c715-4a87-b570-517b831398c0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.796499 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.796475 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.796650 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.796602 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/47b2a6a3-c715-4a87-b570-517b831398c0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.797093 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.797069 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.797290 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.797267 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.797392 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.797347 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.798595 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.798577 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/47b2a6a3-c715-4a87-b570-517b831398c0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.802641 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.802623 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlpsn\" (UniqueName: \"kubernetes.io/projected/47b2a6a3-c715-4a87-b570-517b831398c0-kube-api-access-xlpsn\") pod \"prometheus-k8s-0\" (UID: \"47b2a6a3-c715-4a87-b570-517b831398c0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:14.873942 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:14.873897 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:15.246368 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:15.246343 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 03 10:08:15.248135 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:08:15.248108 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47b2a6a3_c715_4a87_b570_517b831398c0.slice/crio-1adf834fb48839a76ea46fd2c1e6fcf3f73853977b666d10fd3a163358f54359 WatchSource:0}: Error finding container 1adf834fb48839a76ea46fd2c1e6fcf3f73853977b666d10fd3a163358f54359: Status 404 returned error can't find the container with id 1adf834fb48839a76ea46fd2c1e6fcf3f73853977b666d10fd3a163358f54359 Jul 03 10:08:15.461710 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:15.461667 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" event={"ID":"92777b98-f96f-4bb5-927f-5a4b46356a0c","Type":"ContainerStarted","Data":"6d7bc47b22147fa47d1108b82f7ba88038f0509ce4e6c6d0db7e34a572e22dfa"} Jul 03 10:08:15.462173 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:15.461754 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" event={"ID":"92777b98-f96f-4bb5-927f-5a4b46356a0c","Type":"ContainerStarted","Data":"ebe52dc07c9d030324f8406a1b78dfbda6fc7afc69158247a0c87f4f5025c8dc"} Jul 03 10:08:15.462173 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:15.461782 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" event={"ID":"92777b98-f96f-4bb5-927f-5a4b46356a0c","Type":"ContainerStarted","Data":"295519c6b927e9c3291651ec8a5bd36aba7d594ac1fbaabcb5f506f963200c60"} Jul 03 10:08:15.464080 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:15.464056 2573 generic.go:356] "Generic (PLEG): container finished" podID="47b2a6a3-c715-4a87-b570-517b831398c0" containerID="7087a714fb51b8577e36e574f04b00fd54b778ea18ed4e93521df85f59df93ef" exitCode=0 Jul 03 10:08:15.464198 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:15.464105 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"47b2a6a3-c715-4a87-b570-517b831398c0","Type":"ContainerDied","Data":"7087a714fb51b8577e36e574f04b00fd54b778ea18ed4e93521df85f59df93ef"} Jul 03 10:08:15.464198 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:15.464163 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"47b2a6a3-c715-4a87-b570-517b831398c0","Type":"ContainerStarted","Data":"1adf834fb48839a76ea46fd2c1e6fcf3f73853977b666d10fd3a163358f54359"} Jul 03 10:08:15.484637 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:15.484570 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-754fffd6f7-2n7px" podStartSLOduration=1.135938706 podStartE2EDuration="3.484549589s" totalImagesPullingTime="2.348610883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:08:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:08:15.483726942 +0000 UTC m=+150.395616249" watchObservedRunningTime="2026-07-03 10:08:15.484549589 +0000 UTC m=+150.396438894" Jul 03 10:08:16.472409 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:16.472369 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"47b2a6a3-c715-4a87-b570-517b831398c0","Type":"ContainerStarted","Data":"815de5bc4bd82bf7d2ad928f633987618d4bd34c83f7709349915e2672d22dc4"} Jul 03 10:08:16.472870 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:16.472485 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"47b2a6a3-c715-4a87-b570-517b831398c0","Type":"ContainerStarted","Data":"5f1ae4216fd8ff701d7dacc7422ac6bcea15326f9035c3e23a8f3b70f5e93606"} Jul 03 10:08:16.472870 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:16.472535 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"47b2a6a3-c715-4a87-b570-517b831398c0","Type":"ContainerStarted","Data":"74efc97876c897811fea26b1787c8e45e819d7805a68ad987bc9ccdec3380723"} Jul 03 10:08:16.472870 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:16.472577 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"47b2a6a3-c715-4a87-b570-517b831398c0","Type":"ContainerStarted","Data":"4686a363daf500d56d80364adbad2600fee7a6869bfd23172e4f01ee699f8ff3"} Jul 03 10:08:16.472870 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:16.472613 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"47b2a6a3-c715-4a87-b570-517b831398c0","Type":"ContainerStarted","Data":"a561de4fa2d022fb9e0d7a20e00805416a3835e547df7d5caa167f63ef9a5654"} Jul 03 10:08:16.472870 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:16.472661 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"47b2a6a3-c715-4a87-b570-517b831398c0","Type":"ContainerStarted","Data":"936e2e1d38b104da40ec157d8e5efb77b76446f912641234e25aa67ba002da23"} Jul 03 10:08:16.505630 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:16.505567 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.480406657 podStartE2EDuration="2.505547209s" totalImagesPullingTime="0s" totalInitContainerRuntime="25.140552ms" isStatefulPod=false podCreationTimestamp="2026-07-03 10:08:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:08:16.503300507 +0000 UTC m=+151.415189825" watchObservedRunningTime="2026-07-03 10:08:16.505547209 +0000 UTC m=+151.417436513" Jul 03 10:08:19.874685 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:19.874647 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:08:27.470969 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:27.470935 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77849d98d-pxzz4"] Jul 03 10:08:52.490850 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.490787 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-77849d98d-pxzz4" podUID="da235739-fe1e-4e3a-a2d4-e6d786071b13" containerName="console" containerID="cri-o://e132c3d30ed4a7eb021722f555754b875008813ee37a71bffd78a69ff8dc9d60" gracePeriod=15 Jul 03 10:08:52.617849 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.617815 2573 generic.go:356] "Generic (PLEG): container finished" podID="da235739-fe1e-4e3a-a2d4-e6d786071b13" containerID="e132c3d30ed4a7eb021722f555754b875008813ee37a71bffd78a69ff8dc9d60" exitCode=0 Jul 03 10:08:52.617990 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.617884 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77849d98d-pxzz4" event={"ID":"da235739-fe1e-4e3a-a2d4-e6d786071b13","Type":"ContainerDied","Data":"e132c3d30ed4a7eb021722f555754b875008813ee37a71bffd78a69ff8dc9d60"} Jul 03 10:08:52.730831 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.730806 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:08:52.818763 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.818657 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-console-config\") pod \"da235739-fe1e-4e3a-a2d4-e6d786071b13\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " Jul 03 10:08:52.818763 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.818718 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-trusted-ca-bundle\") pod \"da235739-fe1e-4e3a-a2d4-e6d786071b13\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " Jul 03 10:08:52.818976 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.818793 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-service-ca\") pod \"da235739-fe1e-4e3a-a2d4-e6d786071b13\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " Jul 03 10:08:52.818976 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.818861 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-oauth-serving-cert\") pod \"da235739-fe1e-4e3a-a2d4-e6d786071b13\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " Jul 03 10:08:52.818976 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.818901 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8q6k\" (UniqueName: \"kubernetes.io/projected/da235739-fe1e-4e3a-a2d4-e6d786071b13-kube-api-access-l8q6k\") pod \"da235739-fe1e-4e3a-a2d4-e6d786071b13\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " Jul 03 10:08:52.818976 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.818925 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da235739-fe1e-4e3a-a2d4-e6d786071b13-console-serving-cert\") pod \"da235739-fe1e-4e3a-a2d4-e6d786071b13\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " Jul 03 10:08:52.818976 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.818949 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da235739-fe1e-4e3a-a2d4-e6d786071b13-console-oauth-config\") pod \"da235739-fe1e-4e3a-a2d4-e6d786071b13\" (UID: \"da235739-fe1e-4e3a-a2d4-e6d786071b13\") " Jul 03 10:08:52.819267 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.819137 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "da235739-fe1e-4e3a-a2d4-e6d786071b13" (UID: "da235739-fe1e-4e3a-a2d4-e6d786071b13"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:08:52.819330 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.819256 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-service-ca" (OuterVolumeSpecName: "service-ca") pod "da235739-fe1e-4e3a-a2d4-e6d786071b13" (UID: "da235739-fe1e-4e3a-a2d4-e6d786071b13"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:08:52.819330 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.819267 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-console-config" (OuterVolumeSpecName: "console-config") pod "da235739-fe1e-4e3a-a2d4-e6d786071b13" (UID: "da235739-fe1e-4e3a-a2d4-e6d786071b13"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:08:52.819330 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.819279 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "da235739-fe1e-4e3a-a2d4-e6d786071b13" (UID: "da235739-fe1e-4e3a-a2d4-e6d786071b13"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:08:52.821086 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.821054 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da235739-fe1e-4e3a-a2d4-e6d786071b13-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "da235739-fe1e-4e3a-a2d4-e6d786071b13" (UID: "da235739-fe1e-4e3a-a2d4-e6d786071b13"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:08:52.821196 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.821118 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da235739-fe1e-4e3a-a2d4-e6d786071b13-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "da235739-fe1e-4e3a-a2d4-e6d786071b13" (UID: "da235739-fe1e-4e3a-a2d4-e6d786071b13"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:08:52.821196 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.821151 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da235739-fe1e-4e3a-a2d4-e6d786071b13-kube-api-access-l8q6k" (OuterVolumeSpecName: "kube-api-access-l8q6k") pod "da235739-fe1e-4e3a-a2d4-e6d786071b13" (UID: "da235739-fe1e-4e3a-a2d4-e6d786071b13"). InnerVolumeSpecName "kube-api-access-l8q6k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:08:52.919816 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.919777 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-console-config\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:52.919816 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.919808 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-trusted-ca-bundle\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:52.919816 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.919818 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:52.919816 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.919827 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da235739-fe1e-4e3a-a2d4-e6d786071b13-oauth-serving-cert\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:52.920071 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.919836 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8q6k\" (UniqueName: \"kubernetes.io/projected/da235739-fe1e-4e3a-a2d4-e6d786071b13-kube-api-access-l8q6k\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:52.920071 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.919845 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da235739-fe1e-4e3a-a2d4-e6d786071b13-console-serving-cert\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:52.920071 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:52.919853 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da235739-fe1e-4e3a-a2d4-e6d786071b13-console-oauth-config\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:08:53.622412 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:53.622384 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77849d98d-pxzz4" Jul 03 10:08:53.622836 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:53.622387 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77849d98d-pxzz4" event={"ID":"da235739-fe1e-4e3a-a2d4-e6d786071b13","Type":"ContainerDied","Data":"b9949b15191a85ba25abdcebd9c36e1a9bd2f715a1f33878ccbc31466f1a9d10"} Jul 03 10:08:53.622836 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:53.622513 2573 scope.go:117] "RemoveContainer" containerID="e132c3d30ed4a7eb021722f555754b875008813ee37a71bffd78a69ff8dc9d60" Jul 03 10:08:53.644595 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:53.644567 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77849d98d-pxzz4"] Jul 03 10:08:53.649450 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:53.649423 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77849d98d-pxzz4"] Jul 03 10:08:54.562363 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:08:54.562317 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da235739-fe1e-4e3a-a2d4-e6d786071b13" path="/var/lib/kubelet/pods/da235739-fe1e-4e3a-a2d4-e6d786071b13/volumes" Jul 03 10:09:14.874666 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:14.874626 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:09:14.890029 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:14.890003 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:09:15.723558 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:15.723529 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 03 10:09:25.203401 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.203366 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8496857844-lm25b"] Jul 03 10:09:25.203872 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.203689 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da235739-fe1e-4e3a-a2d4-e6d786071b13" containerName="console" Jul 03 10:09:25.203872 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.203701 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="da235739-fe1e-4e3a-a2d4-e6d786071b13" containerName="console" Jul 03 10:09:25.203872 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.203762 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="da235739-fe1e-4e3a-a2d4-e6d786071b13" containerName="console" Jul 03 10:09:25.207663 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.207646 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.211857 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.211835 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 03 10:09:25.213341 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.213036 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 03 10:09:25.213341 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.213105 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-b5rfb\"" Jul 03 10:09:25.213341 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.213171 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 03 10:09:25.213341 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.213212 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 03 10:09:25.213341 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.213308 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 03 10:09:25.213607 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.213447 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 03 10:09:25.213607 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.213488 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 03 10:09:25.219304 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.219285 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 03 10:09:25.222181 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.222151 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8496857844-lm25b"] Jul 03 10:09:25.308001 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.307965 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-service-ca\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.308183 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.308028 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-console-config\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.308183 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.308075 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-oauth-serving-cert\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.308183 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.308106 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbdrz\" (UniqueName: \"kubernetes.io/projected/ae86155d-9312-437d-a521-750997c1927e-kube-api-access-zbdrz\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.308183 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.308160 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae86155d-9312-437d-a521-750997c1927e-console-oauth-config\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.308336 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.308199 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae86155d-9312-437d-a521-750997c1927e-console-serving-cert\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.308336 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.308224 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-trusted-ca-bundle\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.408964 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.408920 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-trusted-ca-bundle\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.408964 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.408966 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-service-ca\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.409217 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.409021 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-console-config\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.409217 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.409081 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-oauth-serving-cert\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.409217 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.409118 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbdrz\" (UniqueName: \"kubernetes.io/projected/ae86155d-9312-437d-a521-750997c1927e-kube-api-access-zbdrz\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.409217 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.409151 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae86155d-9312-437d-a521-750997c1927e-console-oauth-config\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.409217 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.409190 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae86155d-9312-437d-a521-750997c1927e-console-serving-cert\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.409913 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.409887 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-oauth-serving-cert\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.410028 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.409960 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-console-config\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.410028 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.409991 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-service-ca\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.410028 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.410022 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-trusted-ca-bundle\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.411602 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.411584 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae86155d-9312-437d-a521-750997c1927e-console-serving-cert\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.411697 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.411625 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae86155d-9312-437d-a521-750997c1927e-console-oauth-config\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.417523 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.417494 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbdrz\" (UniqueName: \"kubernetes.io/projected/ae86155d-9312-437d-a521-750997c1927e-kube-api-access-zbdrz\") pod \"console-8496857844-lm25b\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.517435 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.517351 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:25.654446 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.654418 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8496857844-lm25b"] Jul 03 10:09:25.656154 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:09:25.656124 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae86155d_9312_437d_a521_750997c1927e.slice/crio-3fdded661b2dfac39eb2bf2a0ddd926b3539d1da25d674b27206d250e464aa6d WatchSource:0}: Error finding container 3fdded661b2dfac39eb2bf2a0ddd926b3539d1da25d674b27206d250e464aa6d: Status 404 returned error can't find the container with id 3fdded661b2dfac39eb2bf2a0ddd926b3539d1da25d674b27206d250e464aa6d Jul 03 10:09:25.751957 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.751921 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8496857844-lm25b" event={"ID":"ae86155d-9312-437d-a521-750997c1927e","Type":"ContainerStarted","Data":"48c412fac93eae7ae5cfab898b9039908cc949b85fc72a87f60f9f2c2b034c94"} Jul 03 10:09:25.751957 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.751967 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8496857844-lm25b" event={"ID":"ae86155d-9312-437d-a521-750997c1927e","Type":"ContainerStarted","Data":"3fdded661b2dfac39eb2bf2a0ddd926b3539d1da25d674b27206d250e464aa6d"} Jul 03 10:09:25.776943 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:25.776830 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8496857844-lm25b" podStartSLOduration=0.77679782 podStartE2EDuration="776.79782ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:09:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:09:25.775664796 +0000 UTC m=+220.687554100" watchObservedRunningTime="2026-07-03 10:09:25.77679782 +0000 UTC m=+220.688687127" Jul 03 10:09:35.517966 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:35.517925 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:35.518335 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:35.517997 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:35.522664 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:35.522642 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8496857844-lm25b" Jul 03 10:09:35.799906 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:09:35.799829 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8496857844-lm25b" Jul 03 10:10:45.488093 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:10:45.488062 2573 kubelet.go:1648] "Image garbage collection succeeded" Jul 03 10:10:50.948109 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:10:50.948076 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-46vcw"] Jul 03 10:10:50.951251 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:10:50.951234 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-46vcw" Jul 03 10:10:50.954245 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:10:50.954213 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 03 10:10:50.954373 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:10:50.954251 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 03 10:10:50.954373 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:10:50.954333 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-bmbmn\"" Jul 03 10:10:50.964830 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:10:50.964810 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-46vcw"] Jul 03 10:10:51.051823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:10:51.051783 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/53f1f829-ddca-4917-ab61-04d089262896-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-46vcw\" (UID: \"53f1f829-ddca-4917-ab61-04d089262896\") " pod="cert-manager/cert-manager-webhook-6486ffd997-46vcw" Jul 03 10:10:51.052006 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:10:51.051844 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g488r\" (UniqueName: \"kubernetes.io/projected/53f1f829-ddca-4917-ab61-04d089262896-kube-api-access-g488r\") pod \"cert-manager-webhook-6486ffd997-46vcw\" (UID: \"53f1f829-ddca-4917-ab61-04d089262896\") " pod="cert-manager/cert-manager-webhook-6486ffd997-46vcw" Jul 03 10:10:51.153335 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:10:51.153292 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/53f1f829-ddca-4917-ab61-04d089262896-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-46vcw\" (UID: \"53f1f829-ddca-4917-ab61-04d089262896\") " pod="cert-manager/cert-manager-webhook-6486ffd997-46vcw" Jul 03 10:10:51.153519 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:10:51.153353 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g488r\" (UniqueName: \"kubernetes.io/projected/53f1f829-ddca-4917-ab61-04d089262896-kube-api-access-g488r\") pod \"cert-manager-webhook-6486ffd997-46vcw\" (UID: \"53f1f829-ddca-4917-ab61-04d089262896\") " pod="cert-manager/cert-manager-webhook-6486ffd997-46vcw" Jul 03 10:10:51.162530 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:10:51.162488 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/53f1f829-ddca-4917-ab61-04d089262896-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-46vcw\" (UID: \"53f1f829-ddca-4917-ab61-04d089262896\") " pod="cert-manager/cert-manager-webhook-6486ffd997-46vcw" Jul 03 10:10:51.162654 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:10:51.162599 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g488r\" (UniqueName: \"kubernetes.io/projected/53f1f829-ddca-4917-ab61-04d089262896-kube-api-access-g488r\") pod \"cert-manager-webhook-6486ffd997-46vcw\" (UID: \"53f1f829-ddca-4917-ab61-04d089262896\") " pod="cert-manager/cert-manager-webhook-6486ffd997-46vcw" Jul 03 10:10:51.279064 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:10:51.278982 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-46vcw" Jul 03 10:10:51.409794 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:10:51.409724 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-46vcw"] Jul 03 10:10:51.412587 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:10:51.412547 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53f1f829_ddca_4917_ab61_04d089262896.slice/crio-6519aa44cee1ff3ce7e39db2374b4d2afa2764753ff6982718f9053bd859800f WatchSource:0}: Error finding container 6519aa44cee1ff3ce7e39db2374b4d2afa2764753ff6982718f9053bd859800f: Status 404 returned error can't find the container with id 6519aa44cee1ff3ce7e39db2374b4d2afa2764753ff6982718f9053bd859800f Jul 03 10:10:51.414532 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:10:51.414517 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 10:10:52.089422 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:10:52.089379 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-46vcw" event={"ID":"53f1f829-ddca-4917-ab61-04d089262896","Type":"ContainerStarted","Data":"6519aa44cee1ff3ce7e39db2374b4d2afa2764753ff6982718f9053bd859800f"} Jul 03 10:10:56.107537 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:10:56.107495 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-46vcw" event={"ID":"53f1f829-ddca-4917-ab61-04d089262896","Type":"ContainerStarted","Data":"73fac4f7624f273d317af992efdd5b71fc154aad857c9af69bc922b1d0fc18c1"} Jul 03 10:10:56.107970 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:10:56.107570 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-46vcw" Jul 03 10:10:56.184345 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:10:56.184292 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-46vcw" podStartSLOduration=2.347854461 podStartE2EDuration="6.184274427s" totalImagesPullingTime="3.836419966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:10:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:10:56.182815579 +0000 UTC m=+311.094704892" watchObservedRunningTime="2026-07-03 10:10:56.184274427 +0000 UTC m=+311.096163730" Jul 03 10:11:00.736466 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:00.736434 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-g768v"] Jul 03 10:11:00.740082 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:00.740064 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-g768v" Jul 03 10:11:00.743797 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:00.743774 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-s986f\"" Jul 03 10:11:00.753500 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:00.753478 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-g768v"] Jul 03 10:11:00.839676 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:00.839635 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmj94\" (UniqueName: \"kubernetes.io/projected/adb80f3c-78a1-4cf9-88c4-10cd6b8dde22-kube-api-access-bmj94\") pod \"cert-manager-55b4d954d5-g768v\" (UID: \"adb80f3c-78a1-4cf9-88c4-10cd6b8dde22\") " pod="cert-manager/cert-manager-55b4d954d5-g768v" Jul 03 10:11:00.839880 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:00.839721 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/adb80f3c-78a1-4cf9-88c4-10cd6b8dde22-bound-sa-token\") pod \"cert-manager-55b4d954d5-g768v\" (UID: \"adb80f3c-78a1-4cf9-88c4-10cd6b8dde22\") " pod="cert-manager/cert-manager-55b4d954d5-g768v" Jul 03 10:11:00.940946 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:00.940901 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/adb80f3c-78a1-4cf9-88c4-10cd6b8dde22-bound-sa-token\") pod \"cert-manager-55b4d954d5-g768v\" (UID: \"adb80f3c-78a1-4cf9-88c4-10cd6b8dde22\") " pod="cert-manager/cert-manager-55b4d954d5-g768v" Jul 03 10:11:00.940946 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:00.940953 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmj94\" (UniqueName: \"kubernetes.io/projected/adb80f3c-78a1-4cf9-88c4-10cd6b8dde22-kube-api-access-bmj94\") pod \"cert-manager-55b4d954d5-g768v\" (UID: \"adb80f3c-78a1-4cf9-88c4-10cd6b8dde22\") " pod="cert-manager/cert-manager-55b4d954d5-g768v" Jul 03 10:11:00.951783 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:00.951749 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmj94\" (UniqueName: \"kubernetes.io/projected/adb80f3c-78a1-4cf9-88c4-10cd6b8dde22-kube-api-access-bmj94\") pod \"cert-manager-55b4d954d5-g768v\" (UID: \"adb80f3c-78a1-4cf9-88c4-10cd6b8dde22\") " pod="cert-manager/cert-manager-55b4d954d5-g768v" Jul 03 10:11:00.951924 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:00.951907 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/adb80f3c-78a1-4cf9-88c4-10cd6b8dde22-bound-sa-token\") pod \"cert-manager-55b4d954d5-g768v\" (UID: \"adb80f3c-78a1-4cf9-88c4-10cd6b8dde22\") " pod="cert-manager/cert-manager-55b4d954d5-g768v" Jul 03 10:11:01.049947 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:01.049839 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-g768v" Jul 03 10:11:01.189488 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:01.189454 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-g768v"] Jul 03 10:11:01.194717 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:11:01.194689 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podadb80f3c_78a1_4cf9_88c4_10cd6b8dde22.slice/crio-4e08121de665b204500a296d38f6da5f4adb2b7962f764ef6e932731b4eb7d61 WatchSource:0}: Error finding container 4e08121de665b204500a296d38f6da5f4adb2b7962f764ef6e932731b4eb7d61: Status 404 returned error can't find the container with id 4e08121de665b204500a296d38f6da5f4adb2b7962f764ef6e932731b4eb7d61 Jul 03 10:11:02.113157 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:02.113126 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-46vcw" Jul 03 10:11:02.131828 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:02.131796 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-g768v" event={"ID":"adb80f3c-78a1-4cf9-88c4-10cd6b8dde22","Type":"ContainerStarted","Data":"533cdb0b9d333658da0ec7cc36fe762c7766095ec5c6a09654176ec78e3d7034"} Jul 03 10:11:02.131978 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:02.131836 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-g768v" event={"ID":"adb80f3c-78a1-4cf9-88c4-10cd6b8dde22","Type":"ContainerStarted","Data":"4e08121de665b204500a296d38f6da5f4adb2b7962f764ef6e932731b4eb7d61"} Jul 03 10:11:02.168515 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:02.168481 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24pn"] Jul 03 10:11:02.169282 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:02.169228 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-g768v" podStartSLOduration=2.16920762 podStartE2EDuration="2.16920762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:11:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:11:02.16912759 +0000 UTC m=+317.081016886" watchObservedRunningTime="2026-07-03 10:11:02.16920762 +0000 UTC m=+317.081096928" Jul 03 10:11:02.172219 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:02.172203 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24pn" Jul 03 10:11:02.176073 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:02.176052 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 03 10:11:02.176610 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:02.176591 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 03 10:11:02.176791 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:02.176772 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-6d4qp\"" Jul 03 10:11:02.185144 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:02.185121 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24pn"] Jul 03 10:11:02.250697 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:02.250665 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f31e0b7b-d1ad-4586-a9ff-e1d7d2e2b5ae-tmp\") pod \"openshift-lws-operator-fd8ccff4c-g24pn\" (UID: \"f31e0b7b-d1ad-4586-a9ff-e1d7d2e2b5ae\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24pn" Jul 03 10:11:02.250697 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:02.250699 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k4xj\" (UniqueName: \"kubernetes.io/projected/f31e0b7b-d1ad-4586-a9ff-e1d7d2e2b5ae-kube-api-access-6k4xj\") pod \"openshift-lws-operator-fd8ccff4c-g24pn\" (UID: \"f31e0b7b-d1ad-4586-a9ff-e1d7d2e2b5ae\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24pn" Jul 03 10:11:02.352193 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:02.352155 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f31e0b7b-d1ad-4586-a9ff-e1d7d2e2b5ae-tmp\") pod \"openshift-lws-operator-fd8ccff4c-g24pn\" (UID: \"f31e0b7b-d1ad-4586-a9ff-e1d7d2e2b5ae\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24pn" Jul 03 10:11:02.352193 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:02.352196 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k4xj\" (UniqueName: \"kubernetes.io/projected/f31e0b7b-d1ad-4586-a9ff-e1d7d2e2b5ae-kube-api-access-6k4xj\") pod \"openshift-lws-operator-fd8ccff4c-g24pn\" (UID: \"f31e0b7b-d1ad-4586-a9ff-e1d7d2e2b5ae\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24pn" Jul 03 10:11:02.352548 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:02.352528 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f31e0b7b-d1ad-4586-a9ff-e1d7d2e2b5ae-tmp\") pod \"openshift-lws-operator-fd8ccff4c-g24pn\" (UID: \"f31e0b7b-d1ad-4586-a9ff-e1d7d2e2b5ae\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24pn" Jul 03 10:11:02.363133 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:02.363103 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k4xj\" (UniqueName: \"kubernetes.io/projected/f31e0b7b-d1ad-4586-a9ff-e1d7d2e2b5ae-kube-api-access-6k4xj\") pod \"openshift-lws-operator-fd8ccff4c-g24pn\" (UID: \"f31e0b7b-d1ad-4586-a9ff-e1d7d2e2b5ae\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24pn" Jul 03 10:11:02.482996 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:02.482949 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24pn" Jul 03 10:11:02.611181 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:02.611073 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24pn"] Jul 03 10:11:02.613488 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:11:02.613459 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf31e0b7b_d1ad_4586_a9ff_e1d7d2e2b5ae.slice/crio-ff6f6b05e2b12be4876fb47fee2f3e2ffef6521f78fbe039db9875b0cc231f29 WatchSource:0}: Error finding container ff6f6b05e2b12be4876fb47fee2f3e2ffef6521f78fbe039db9875b0cc231f29: Status 404 returned error can't find the container with id ff6f6b05e2b12be4876fb47fee2f3e2ffef6521f78fbe039db9875b0cc231f29 Jul 03 10:11:03.137039 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:03.136997 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24pn" event={"ID":"f31e0b7b-d1ad-4586-a9ff-e1d7d2e2b5ae","Type":"ContainerStarted","Data":"ff6f6b05e2b12be4876fb47fee2f3e2ffef6521f78fbe039db9875b0cc231f29"} Jul 03 10:11:05.147994 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:05.147958 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24pn" event={"ID":"f31e0b7b-d1ad-4586-a9ff-e1d7d2e2b5ae","Type":"ContainerStarted","Data":"4dd0c436c6e9486f268c60603f33058883f85fb74cc258f48efa29c89cccc067"} Jul 03 10:11:05.168501 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:05.168447 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-g24pn" podStartSLOduration=0.805643417 podStartE2EDuration="3.168427359s" totalImagesPullingTime="2.362783942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:11:05.165534506 +0000 UTC m=+320.077423810" watchObservedRunningTime="2026-07-03 10:11:05.168427359 +0000 UTC m=+320.080316665" Jul 03 10:11:21.614647 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.614603 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r"] Jul 03 10:11:21.621147 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.621119 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" Jul 03 10:11:21.633355 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.633338 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 03 10:11:21.633623 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.633604 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 03 10:11:21.633711 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.633633 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 03 10:11:21.634790 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.634772 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 03 10:11:21.634898 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.634809 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hhdl4\"" Jul 03 10:11:21.702700 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.702665 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r"] Jul 03 10:11:21.726792 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.726723 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3c088a64-a749-482a-a253-596c1345f9cc-webhook-cert\") pod \"opendatahub-operator-controller-manager-789547c8f-4pl9r\" (UID: \"3c088a64-a749-482a-a253-596c1345f9cc\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" Jul 03 10:11:21.726954 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.726792 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3c088a64-a749-482a-a253-596c1345f9cc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-789547c8f-4pl9r\" (UID: \"3c088a64-a749-482a-a253-596c1345f9cc\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" Jul 03 10:11:21.726954 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.726821 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jkq8\" (UniqueName: \"kubernetes.io/projected/3c088a64-a749-482a-a253-596c1345f9cc-kube-api-access-5jkq8\") pod \"opendatahub-operator-controller-manager-789547c8f-4pl9r\" (UID: \"3c088a64-a749-482a-a253-596c1345f9cc\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" Jul 03 10:11:21.726954 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.726868 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c088a64-a749-482a-a253-596c1345f9cc-tmp\") pod \"opendatahub-operator-controller-manager-789547c8f-4pl9r\" (UID: \"3c088a64-a749-482a-a253-596c1345f9cc\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" Jul 03 10:11:21.727057 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.726962 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3c088a64-a749-482a-a253-596c1345f9cc-manifests\") pod \"opendatahub-operator-controller-manager-789547c8f-4pl9r\" (UID: \"3c088a64-a749-482a-a253-596c1345f9cc\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" Jul 03 10:11:21.827547 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.827511 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3c088a64-a749-482a-a253-596c1345f9cc-webhook-cert\") pod \"opendatahub-operator-controller-manager-789547c8f-4pl9r\" (UID: \"3c088a64-a749-482a-a253-596c1345f9cc\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" Jul 03 10:11:21.827712 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.827549 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3c088a64-a749-482a-a253-596c1345f9cc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-789547c8f-4pl9r\" (UID: \"3c088a64-a749-482a-a253-596c1345f9cc\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" Jul 03 10:11:21.827712 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.827676 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jkq8\" (UniqueName: \"kubernetes.io/projected/3c088a64-a749-482a-a253-596c1345f9cc-kube-api-access-5jkq8\") pod \"opendatahub-operator-controller-manager-789547c8f-4pl9r\" (UID: \"3c088a64-a749-482a-a253-596c1345f9cc\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" Jul 03 10:11:21.827843 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.827779 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c088a64-a749-482a-a253-596c1345f9cc-tmp\") pod \"opendatahub-operator-controller-manager-789547c8f-4pl9r\" (UID: \"3c088a64-a749-482a-a253-596c1345f9cc\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" Jul 03 10:11:21.827904 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.827877 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3c088a64-a749-482a-a253-596c1345f9cc-manifests\") pod \"opendatahub-operator-controller-manager-789547c8f-4pl9r\" (UID: \"3c088a64-a749-482a-a253-596c1345f9cc\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" Jul 03 10:11:21.828318 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.828295 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3c088a64-a749-482a-a253-596c1345f9cc-manifests\") pod \"opendatahub-operator-controller-manager-789547c8f-4pl9r\" (UID: \"3c088a64-a749-482a-a253-596c1345f9cc\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" Jul 03 10:11:21.830187 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.830126 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3c088a64-a749-482a-a253-596c1345f9cc-tmp\") pod \"opendatahub-operator-controller-manager-789547c8f-4pl9r\" (UID: \"3c088a64-a749-482a-a253-596c1345f9cc\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" Jul 03 10:11:21.830648 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.830625 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3c088a64-a749-482a-a253-596c1345f9cc-webhook-cert\") pod \"opendatahub-operator-controller-manager-789547c8f-4pl9r\" (UID: \"3c088a64-a749-482a-a253-596c1345f9cc\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" Jul 03 10:11:21.830752 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.830630 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3c088a64-a749-482a-a253-596c1345f9cc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-789547c8f-4pl9r\" (UID: \"3c088a64-a749-482a-a253-596c1345f9cc\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" Jul 03 10:11:21.843550 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.843525 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jkq8\" (UniqueName: \"kubernetes.io/projected/3c088a64-a749-482a-a253-596c1345f9cc-kube-api-access-5jkq8\") pod \"opendatahub-operator-controller-manager-789547c8f-4pl9r\" (UID: \"3c088a64-a749-482a-a253-596c1345f9cc\") " pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" Jul 03 10:11:21.936321 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:21.936288 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" Jul 03 10:11:22.127016 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:22.126864 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r"] Jul 03 10:11:22.129796 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:11:22.129763 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c088a64_a749_482a_a253_596c1345f9cc.slice/crio-aa7f510e50b407d780c5dd9d88dcce54f4232f5a171bd32c0980b27d77c2dce9 WatchSource:0}: Error finding container aa7f510e50b407d780c5dd9d88dcce54f4232f5a171bd32c0980b27d77c2dce9: Status 404 returned error can't find the container with id aa7f510e50b407d780c5dd9d88dcce54f4232f5a171bd32c0980b27d77c2dce9 Jul 03 10:11:22.218794 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:22.218698 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" event={"ID":"3c088a64-a749-482a-a253-596c1345f9cc","Type":"ContainerStarted","Data":"aa7f510e50b407d780c5dd9d88dcce54f4232f5a171bd32c0980b27d77c2dce9"} Jul 03 10:11:26.239725 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:26.239684 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" event={"ID":"3c088a64-a749-482a-a253-596c1345f9cc","Type":"ContainerStarted","Data":"2f8a44f408c6e09f7d53ffdb1f8dabc0a65741e5f068925c0c546f6f4fdd101b"} Jul 03 10:11:27.246792 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:27.246754 2573 generic.go:356] "Generic (PLEG): container finished" podID="3c088a64-a749-482a-a253-596c1345f9cc" containerID="2f8a44f408c6e09f7d53ffdb1f8dabc0a65741e5f068925c0c546f6f4fdd101b" exitCode=0 Jul 03 10:11:27.247236 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:27.246820 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" event={"ID":"3c088a64-a749-482a-a253-596c1345f9cc","Type":"ContainerDied","Data":"2f8a44f408c6e09f7d53ffdb1f8dabc0a65741e5f068925c0c546f6f4fdd101b"} Jul 03 10:11:28.253816 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:28.253775 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" event={"ID":"3c088a64-a749-482a-a253-596c1345f9cc","Type":"ContainerStarted","Data":"18c6d5203ee8ff99f37daee83b629044a57907ec32c3699c1769257c555070cd"} Jul 03 10:11:28.254299 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:28.253879 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" Jul 03 10:11:28.283028 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:28.282974 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" podStartSLOduration=2.620296882 podStartE2EDuration="7.282957438s" totalImagesPullingTime="3.562567282s" totalInitContainerRuntime="1.100093274s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:11:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:11:28.281144113 +0000 UTC m=+343.193033428" watchObservedRunningTime="2026-07-03 10:11:28.282957438 +0000 UTC m=+343.194846742" Jul 03 10:11:39.261461 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:39.261379 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-789547c8f-4pl9r" Jul 03 10:11:40.975821 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:40.975788 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9"] Jul 03 10:11:40.978651 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:40.978630 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9" Jul 03 10:11:40.983971 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:40.983948 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 03 10:11:40.984094 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:40.984022 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 03 10:11:40.984159 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:40.984141 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 03 10:11:40.984159 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:40.983985 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-d6g86\"" Jul 03 10:11:41.017508 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.017473 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9"] Jul 03 10:11:41.113035 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.112998 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a261ae40-ff24-43d2-ae42-1f8cb65c2304-metrics-cert\") pod \"lws-controller-manager-5779fb96bf-qkbc9\" (UID: \"a261ae40-ff24-43d2-ae42-1f8cb65c2304\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9" Jul 03 10:11:41.113035 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.113035 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4dz6\" (UniqueName: \"kubernetes.io/projected/a261ae40-ff24-43d2-ae42-1f8cb65c2304-kube-api-access-c4dz6\") pod \"lws-controller-manager-5779fb96bf-qkbc9\" (UID: \"a261ae40-ff24-43d2-ae42-1f8cb65c2304\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9" Jul 03 10:11:41.113264 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.113068 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a261ae40-ff24-43d2-ae42-1f8cb65c2304-manager-config\") pod \"lws-controller-manager-5779fb96bf-qkbc9\" (UID: \"a261ae40-ff24-43d2-ae42-1f8cb65c2304\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9" Jul 03 10:11:41.113264 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.113143 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a261ae40-ff24-43d2-ae42-1f8cb65c2304-cert\") pod \"lws-controller-manager-5779fb96bf-qkbc9\" (UID: \"a261ae40-ff24-43d2-ae42-1f8cb65c2304\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9" Jul 03 10:11:41.214254 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.214216 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a261ae40-ff24-43d2-ae42-1f8cb65c2304-metrics-cert\") pod \"lws-controller-manager-5779fb96bf-qkbc9\" (UID: \"a261ae40-ff24-43d2-ae42-1f8cb65c2304\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9" Jul 03 10:11:41.214254 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.214253 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4dz6\" (UniqueName: \"kubernetes.io/projected/a261ae40-ff24-43d2-ae42-1f8cb65c2304-kube-api-access-c4dz6\") pod \"lws-controller-manager-5779fb96bf-qkbc9\" (UID: \"a261ae40-ff24-43d2-ae42-1f8cb65c2304\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9" Jul 03 10:11:41.214477 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.214382 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a261ae40-ff24-43d2-ae42-1f8cb65c2304-manager-config\") pod \"lws-controller-manager-5779fb96bf-qkbc9\" (UID: \"a261ae40-ff24-43d2-ae42-1f8cb65c2304\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9" Jul 03 10:11:41.214477 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.214425 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a261ae40-ff24-43d2-ae42-1f8cb65c2304-cert\") pod \"lws-controller-manager-5779fb96bf-qkbc9\" (UID: \"a261ae40-ff24-43d2-ae42-1f8cb65c2304\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9" Jul 03 10:11:41.215103 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.215077 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a261ae40-ff24-43d2-ae42-1f8cb65c2304-manager-config\") pod \"lws-controller-manager-5779fb96bf-qkbc9\" (UID: \"a261ae40-ff24-43d2-ae42-1f8cb65c2304\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9" Jul 03 10:11:41.216613 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.216575 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a261ae40-ff24-43d2-ae42-1f8cb65c2304-metrics-cert\") pod \"lws-controller-manager-5779fb96bf-qkbc9\" (UID: \"a261ae40-ff24-43d2-ae42-1f8cb65c2304\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9" Jul 03 10:11:41.216824 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.216808 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a261ae40-ff24-43d2-ae42-1f8cb65c2304-cert\") pod \"lws-controller-manager-5779fb96bf-qkbc9\" (UID: \"a261ae40-ff24-43d2-ae42-1f8cb65c2304\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9" Jul 03 10:11:41.236280 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.236216 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4dz6\" (UniqueName: \"kubernetes.io/projected/a261ae40-ff24-43d2-ae42-1f8cb65c2304-kube-api-access-c4dz6\") pod \"lws-controller-manager-5779fb96bf-qkbc9\" (UID: \"a261ae40-ff24-43d2-ae42-1f8cb65c2304\") " pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9" Jul 03 10:11:41.288242 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.288214 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9" Jul 03 10:11:41.441799 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.441764 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9"] Jul 03 10:11:41.445066 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:11:41.445027 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda261ae40_ff24_43d2_ae42_1f8cb65c2304.slice/crio-a103904a041bae57267089a03e239660da8b04856eafbe6e98ced32cfe9659ca WatchSource:0}: Error finding container a103904a041bae57267089a03e239660da8b04856eafbe6e98ced32cfe9659ca: Status 404 returned error can't find the container with id a103904a041bae57267089a03e239660da8b04856eafbe6e98ced32cfe9659ca Jul 03 10:11:41.590868 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.590789 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b56494769-5khf6"] Jul 03 10:11:41.595376 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.595358 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b56494769-5khf6" Jul 03 10:11:41.600210 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.600151 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 03 10:11:41.601109 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.601088 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 03 10:11:41.601229 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.601116 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-wdsdl\"" Jul 03 10:11:41.601460 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.601438 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 03 10:11:41.603217 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.603199 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 03 10:11:41.618681 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.618651 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b56494769-5khf6"] Jul 03 10:11:41.719966 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.719925 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcbsk\" (UniqueName: \"kubernetes.io/projected/6802d445-8d73-4ccd-8ce4-1b6b30d6708b-kube-api-access-qcbsk\") pod \"kube-auth-proxy-5b56494769-5khf6\" (UID: \"6802d445-8d73-4ccd-8ce4-1b6b30d6708b\") " pod="openshift-ingress/kube-auth-proxy-5b56494769-5khf6" Jul 03 10:11:41.720153 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.719985 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6802d445-8d73-4ccd-8ce4-1b6b30d6708b-tls-certs\") pod \"kube-auth-proxy-5b56494769-5khf6\" (UID: \"6802d445-8d73-4ccd-8ce4-1b6b30d6708b\") " pod="openshift-ingress/kube-auth-proxy-5b56494769-5khf6" Jul 03 10:11:41.720153 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.720046 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6802d445-8d73-4ccd-8ce4-1b6b30d6708b-tmp\") pod \"kube-auth-proxy-5b56494769-5khf6\" (UID: \"6802d445-8d73-4ccd-8ce4-1b6b30d6708b\") " pod="openshift-ingress/kube-auth-proxy-5b56494769-5khf6" Jul 03 10:11:41.821236 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.821193 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6802d445-8d73-4ccd-8ce4-1b6b30d6708b-tls-certs\") pod \"kube-auth-proxy-5b56494769-5khf6\" (UID: \"6802d445-8d73-4ccd-8ce4-1b6b30d6708b\") " pod="openshift-ingress/kube-auth-proxy-5b56494769-5khf6" Jul 03 10:11:41.821236 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.821238 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6802d445-8d73-4ccd-8ce4-1b6b30d6708b-tmp\") pod \"kube-auth-proxy-5b56494769-5khf6\" (UID: \"6802d445-8d73-4ccd-8ce4-1b6b30d6708b\") " pod="openshift-ingress/kube-auth-proxy-5b56494769-5khf6" Jul 03 10:11:41.821457 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.821303 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcbsk\" (UniqueName: \"kubernetes.io/projected/6802d445-8d73-4ccd-8ce4-1b6b30d6708b-kube-api-access-qcbsk\") pod \"kube-auth-proxy-5b56494769-5khf6\" (UID: \"6802d445-8d73-4ccd-8ce4-1b6b30d6708b\") " pod="openshift-ingress/kube-auth-proxy-5b56494769-5khf6" Jul 03 10:11:41.821457 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:11:41.821336 2573 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 03 10:11:41.821457 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:11:41.821410 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6802d445-8d73-4ccd-8ce4-1b6b30d6708b-tls-certs podName:6802d445-8d73-4ccd-8ce4-1b6b30d6708b nodeName:}" failed. No retries permitted until 2026-07-03 10:11:42.321387316 +0000 UTC m=+357.233276597 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/6802d445-8d73-4ccd-8ce4-1b6b30d6708b-tls-certs") pod "kube-auth-proxy-5b56494769-5khf6" (UID: "6802d445-8d73-4ccd-8ce4-1b6b30d6708b") : secret "kube-auth-proxy-tls" not found Jul 03 10:11:41.823533 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.823516 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6802d445-8d73-4ccd-8ce4-1b6b30d6708b-tmp\") pod \"kube-auth-proxy-5b56494769-5khf6\" (UID: \"6802d445-8d73-4ccd-8ce4-1b6b30d6708b\") " pod="openshift-ingress/kube-auth-proxy-5b56494769-5khf6" Jul 03 10:11:41.835005 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:41.834981 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcbsk\" (UniqueName: \"kubernetes.io/projected/6802d445-8d73-4ccd-8ce4-1b6b30d6708b-kube-api-access-qcbsk\") pod \"kube-auth-proxy-5b56494769-5khf6\" (UID: \"6802d445-8d73-4ccd-8ce4-1b6b30d6708b\") " pod="openshift-ingress/kube-auth-proxy-5b56494769-5khf6" Jul 03 10:11:42.316861 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:42.316822 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9" event={"ID":"a261ae40-ff24-43d2-ae42-1f8cb65c2304","Type":"ContainerStarted","Data":"a103904a041bae57267089a03e239660da8b04856eafbe6e98ced32cfe9659ca"} Jul 03 10:11:42.326230 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:42.326204 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6802d445-8d73-4ccd-8ce4-1b6b30d6708b-tls-certs\") pod \"kube-auth-proxy-5b56494769-5khf6\" (UID: \"6802d445-8d73-4ccd-8ce4-1b6b30d6708b\") " pod="openshift-ingress/kube-auth-proxy-5b56494769-5khf6" Jul 03 10:11:42.328532 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:42.328508 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6802d445-8d73-4ccd-8ce4-1b6b30d6708b-tls-certs\") pod \"kube-auth-proxy-5b56494769-5khf6\" (UID: \"6802d445-8d73-4ccd-8ce4-1b6b30d6708b\") " pod="openshift-ingress/kube-auth-proxy-5b56494769-5khf6" Jul 03 10:11:42.505349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:42.505309 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b56494769-5khf6" Jul 03 10:11:42.693391 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:42.693342 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b56494769-5khf6"] Jul 03 10:11:42.695245 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:11:42.694601 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6802d445_8d73_4ccd_8ce4_1b6b30d6708b.slice/crio-c0268ec7a034c683dc73b81e97a32b863225fe48316384665a59d2c61195bb42 WatchSource:0}: Error finding container c0268ec7a034c683dc73b81e97a32b863225fe48316384665a59d2c61195bb42: Status 404 returned error can't find the container with id c0268ec7a034c683dc73b81e97a32b863225fe48316384665a59d2c61195bb42 Jul 03 10:11:43.328321 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:43.328279 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b56494769-5khf6" event={"ID":"6802d445-8d73-4ccd-8ce4-1b6b30d6708b","Type":"ContainerStarted","Data":"c0268ec7a034c683dc73b81e97a32b863225fe48316384665a59d2c61195bb42"} Jul 03 10:11:44.334694 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:44.334660 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9" event={"ID":"a261ae40-ff24-43d2-ae42-1f8cb65c2304","Type":"ContainerStarted","Data":"78139ad767ac2cf9987b0f794d93c8e363603afbab5c728d82f07d622c904031"} Jul 03 10:11:44.335102 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:44.334791 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9" Jul 03 10:11:44.353770 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:44.353701 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9" podStartSLOduration=2.260755229 podStartE2EDuration="4.353685618s" totalImagesPullingTime="2.092930389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:11:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:11:44.351942967 +0000 UTC m=+359.263832272" watchObservedRunningTime="2026-07-03 10:11:44.353685618 +0000 UTC m=+359.265574926" Jul 03 10:11:45.595483 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:45.595463 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 03 10:11:46.345896 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:46.345860 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b56494769-5khf6" event={"ID":"6802d445-8d73-4ccd-8ce4-1b6b30d6708b","Type":"ContainerStarted","Data":"dd61859374316a9c33476745da4ff22b063cc739bd886df0f862111aa3479a6f"} Jul 03 10:11:46.363706 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:46.363661 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b56494769-5khf6" podStartSLOduration=2.467969313 podStartE2EDuration="5.36364121s" totalImagesPullingTime="2.895671897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:11:46.362972686 +0000 UTC m=+361.274861991" watchObservedRunningTime="2026-07-03 10:11:46.36364121 +0000 UTC m=+361.275530514" Jul 03 10:11:55.343245 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:11:55.343206 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5779fb96bf-qkbc9" Jul 03 10:13:21.915638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:21.915563 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6846446956-mzffw"] Jul 03 10:13:21.919058 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:21.919037 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:21.934728 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:21.934703 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6846446956-mzffw"] Jul 03 10:13:22.019813 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.019776 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c55b2578-0178-406c-8534-3279badb97c0-console-config\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.019813 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.019813 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c55b2578-0178-406c-8534-3279badb97c0-trusted-ca-bundle\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.020034 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.019838 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c55b2578-0178-406c-8534-3279badb97c0-oauth-serving-cert\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.020034 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.019946 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c55b2578-0178-406c-8534-3279badb97c0-console-serving-cert\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.020034 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.019986 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c55b2578-0178-406c-8534-3279badb97c0-console-oauth-config\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.020128 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.020053 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c55b2578-0178-406c-8534-3279badb97c0-service-ca\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.020128 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.020070 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbp9s\" (UniqueName: \"kubernetes.io/projected/c55b2578-0178-406c-8534-3279badb97c0-kube-api-access-dbp9s\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.120817 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.120778 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c55b2578-0178-406c-8534-3279badb97c0-console-config\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.120817 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.120818 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c55b2578-0178-406c-8534-3279badb97c0-trusted-ca-bundle\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.121044 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.120840 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c55b2578-0178-406c-8534-3279badb97c0-oauth-serving-cert\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.121044 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.120892 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c55b2578-0178-406c-8534-3279badb97c0-console-serving-cert\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.121044 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.120919 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c55b2578-0178-406c-8534-3279badb97c0-console-oauth-config\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.121044 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.120973 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c55b2578-0178-406c-8534-3279badb97c0-service-ca\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.121044 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.120997 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbp9s\" (UniqueName: \"kubernetes.io/projected/c55b2578-0178-406c-8534-3279badb97c0-kube-api-access-dbp9s\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.121642 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.121606 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c55b2578-0178-406c-8534-3279badb97c0-console-config\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.121788 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.121657 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c55b2578-0178-406c-8534-3279badb97c0-oauth-serving-cert\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.121788 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.121669 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c55b2578-0178-406c-8534-3279badb97c0-service-ca\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.121969 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.121952 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c55b2578-0178-406c-8534-3279badb97c0-trusted-ca-bundle\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.123277 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.123259 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c55b2578-0178-406c-8534-3279badb97c0-console-oauth-config\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.123402 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.123377 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c55b2578-0178-406c-8534-3279badb97c0-console-serving-cert\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.141192 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.141167 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbp9s\" (UniqueName: \"kubernetes.io/projected/c55b2578-0178-406c-8534-3279badb97c0-kube-api-access-dbp9s\") pod \"console-6846446956-mzffw\" (UID: \"c55b2578-0178-406c-8534-3279badb97c0\") " pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.228195 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.228103 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:22.369526 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.369502 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6846446956-mzffw"] Jul 03 10:13:22.371744 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:13:22.371708 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc55b2578_0178_406c_8534_3279badb97c0.slice/crio-4b2dd45288b4dd94860a873577d2cc1de674f8b210b83470396e0a626bb77481 WatchSource:0}: Error finding container 4b2dd45288b4dd94860a873577d2cc1de674f8b210b83470396e0a626bb77481: Status 404 returned error can't find the container with id 4b2dd45288b4dd94860a873577d2cc1de674f8b210b83470396e0a626bb77481 Jul 03 10:13:22.792254 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.792220 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6846446956-mzffw" event={"ID":"c55b2578-0178-406c-8534-3279badb97c0","Type":"ContainerStarted","Data":"945e13bb2e491cb74cdaf724bf140f7957c1a441672a0b12fe8f2166a98e2aa7"} Jul 03 10:13:22.792426 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.792271 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6846446956-mzffw" event={"ID":"c55b2578-0178-406c-8534-3279badb97c0","Type":"ContainerStarted","Data":"4b2dd45288b4dd94860a873577d2cc1de674f8b210b83470396e0a626bb77481"} Jul 03 10:13:22.814397 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:22.814349 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6846446956-mzffw" podStartSLOduration=1.814334806 podStartE2EDuration="1.814334806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:13:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:13:22.812722855 +0000 UTC m=+457.724612183" watchObservedRunningTime="2026-07-03 10:13:22.814334806 +0000 UTC m=+457.726224109" Jul 03 10:13:24.913744 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:24.913695 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-kxcwc"] Jul 03 10:13:24.917052 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:24.917035 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kxcwc" Jul 03 10:13:24.920172 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:24.920148 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 03 10:13:24.920172 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:24.920167 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 03 10:13:24.921599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:24.921583 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-j7zw7\"" Jul 03 10:13:24.921674 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:24.921621 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 03 10:13:24.927366 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:24.927347 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-kxcwc"] Jul 03 10:13:24.948042 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:24.948013 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8vsq\" (UniqueName: \"kubernetes.io/projected/eefa695e-b755-4069-8637-2837891c55b5-kube-api-access-f8vsq\") pod \"dns-operator-controller-manager-864dd78c4c-kxcwc\" (UID: \"eefa695e-b755-4069-8637-2837891c55b5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kxcwc" Jul 03 10:13:25.049585 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:25.049523 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8vsq\" (UniqueName: \"kubernetes.io/projected/eefa695e-b755-4069-8637-2837891c55b5-kube-api-access-f8vsq\") pod \"dns-operator-controller-manager-864dd78c4c-kxcwc\" (UID: \"eefa695e-b755-4069-8637-2837891c55b5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kxcwc" Jul 03 10:13:25.062697 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:25.062669 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8vsq\" (UniqueName: \"kubernetes.io/projected/eefa695e-b755-4069-8637-2837891c55b5-kube-api-access-f8vsq\") pod \"dns-operator-controller-manager-864dd78c4c-kxcwc\" (UID: \"eefa695e-b755-4069-8637-2837891c55b5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kxcwc" Jul 03 10:13:25.227805 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:25.227695 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kxcwc" Jul 03 10:13:25.365505 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:25.365472 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-kxcwc"] Jul 03 10:13:25.368998 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:13:25.368967 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeefa695e_b755_4069_8637_2837891c55b5.slice/crio-22d350d9f641360321a5eefa04c732d7c3e0713dbfe1fe3615ba2bdf41b62933 WatchSource:0}: Error finding container 22d350d9f641360321a5eefa04c732d7c3e0713dbfe1fe3615ba2bdf41b62933: Status 404 returned error can't find the container with id 22d350d9f641360321a5eefa04c732d7c3e0713dbfe1fe3615ba2bdf41b62933 Jul 03 10:13:25.807219 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:25.807187 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kxcwc" event={"ID":"eefa695e-b755-4069-8637-2837891c55b5","Type":"ContainerStarted","Data":"22d350d9f641360321a5eefa04c732d7c3e0713dbfe1fe3615ba2bdf41b62933"} Jul 03 10:13:28.822867 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:28.822772 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kxcwc" event={"ID":"eefa695e-b755-4069-8637-2837891c55b5","Type":"ContainerStarted","Data":"9dcb363c7488c03050502e16dcd4a28700fed5ad8c0ae322069d438777e8b618"} Jul 03 10:13:28.823251 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:28.822952 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kxcwc" Jul 03 10:13:28.855696 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:28.855636 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kxcwc" podStartSLOduration=1.73221063 podStartE2EDuration="4.855618881s" totalImagesPullingTime="3.123408251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:13:28.854706582 +0000 UTC m=+463.766595889" watchObservedRunningTime="2026-07-03 10:13:28.855618881 +0000 UTC m=+463.767508185" Jul 03 10:13:32.228399 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:32.228359 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:32.228842 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:32.228449 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:32.233224 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:32.233203 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:32.844389 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:32.844362 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6846446956-mzffw" Jul 03 10:13:32.926223 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:32.926193 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8496857844-lm25b"] Jul 03 10:13:39.830094 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:39.830058 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kxcwc" Jul 03 10:13:57.946514 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:57.946449 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-8496857844-lm25b" podUID="ae86155d-9312-437d-a521-750997c1927e" containerName="console" containerID="cri-o://48c412fac93eae7ae5cfab898b9039908cc949b85fc72a87f60f9f2c2b034c94" gracePeriod=15 Jul 03 10:13:58.198179 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.198121 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8496857844-lm25b" Jul 03 10:13:58.255146 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.255115 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-trusted-ca-bundle\") pod \"ae86155d-9312-437d-a521-750997c1927e\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " Jul 03 10:13:58.255296 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.255158 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-service-ca\") pod \"ae86155d-9312-437d-a521-750997c1927e\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " Jul 03 10:13:58.255296 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.255187 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-console-config\") pod \"ae86155d-9312-437d-a521-750997c1927e\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " Jul 03 10:13:58.255423 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.255334 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae86155d-9312-437d-a521-750997c1927e-console-serving-cert\") pod \"ae86155d-9312-437d-a521-750997c1927e\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " Jul 03 10:13:58.255423 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.255379 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae86155d-9312-437d-a521-750997c1927e-console-oauth-config\") pod \"ae86155d-9312-437d-a521-750997c1927e\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " Jul 03 10:13:58.255526 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.255448 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-oauth-serving-cert\") pod \"ae86155d-9312-437d-a521-750997c1927e\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " Jul 03 10:13:58.255526 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.255484 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbdrz\" (UniqueName: \"kubernetes.io/projected/ae86155d-9312-437d-a521-750997c1927e-kube-api-access-zbdrz\") pod \"ae86155d-9312-437d-a521-750997c1927e\" (UID: \"ae86155d-9312-437d-a521-750997c1927e\") " Jul 03 10:13:58.255617 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.255579 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-console-config" (OuterVolumeSpecName: "console-config") pod "ae86155d-9312-437d-a521-750997c1927e" (UID: "ae86155d-9312-437d-a521-750997c1927e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:13:58.255791 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.255766 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-service-ca" (OuterVolumeSpecName: "service-ca") pod "ae86155d-9312-437d-a521-750997c1927e" (UID: "ae86155d-9312-437d-a521-750997c1927e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:13:58.255791 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.255782 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ae86155d-9312-437d-a521-750997c1927e" (UID: "ae86155d-9312-437d-a521-750997c1927e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:13:58.255913 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.255833 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ae86155d-9312-437d-a521-750997c1927e" (UID: "ae86155d-9312-437d-a521-750997c1927e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:13:58.255913 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.255892 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-oauth-serving-cert\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:13:58.255913 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.255905 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-trusted-ca-bundle\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:13:58.255913 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.255914 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:13:58.256110 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.255922 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae86155d-9312-437d-a521-750997c1927e-console-config\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:13:58.257534 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.257500 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae86155d-9312-437d-a521-750997c1927e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ae86155d-9312-437d-a521-750997c1927e" (UID: "ae86155d-9312-437d-a521-750997c1927e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:13:58.257644 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.257560 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae86155d-9312-437d-a521-750997c1927e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ae86155d-9312-437d-a521-750997c1927e" (UID: "ae86155d-9312-437d-a521-750997c1927e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:13:58.257644 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.257578 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae86155d-9312-437d-a521-750997c1927e-kube-api-access-zbdrz" (OuterVolumeSpecName: "kube-api-access-zbdrz") pod "ae86155d-9312-437d-a521-750997c1927e" (UID: "ae86155d-9312-437d-a521-750997c1927e"). InnerVolumeSpecName "kube-api-access-zbdrz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:13:58.356704 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.356674 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae86155d-9312-437d-a521-750997c1927e-console-serving-cert\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:13:58.356704 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.356699 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae86155d-9312-437d-a521-750997c1927e-console-oauth-config\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:13:58.356704 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.356710 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbdrz\" (UniqueName: \"kubernetes.io/projected/ae86155d-9312-437d-a521-750997c1927e-kube-api-access-zbdrz\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:13:58.962262 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.962227 2573 generic.go:356] "Generic (PLEG): container finished" podID="ae86155d-9312-437d-a521-750997c1927e" containerID="48c412fac93eae7ae5cfab898b9039908cc949b85fc72a87f60f9f2c2b034c94" exitCode=0 Jul 03 10:13:58.962720 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.962269 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8496857844-lm25b" Jul 03 10:13:58.962720 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.962298 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8496857844-lm25b" event={"ID":"ae86155d-9312-437d-a521-750997c1927e","Type":"ContainerDied","Data":"48c412fac93eae7ae5cfab898b9039908cc949b85fc72a87f60f9f2c2b034c94"} Jul 03 10:13:58.962720 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.962348 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8496857844-lm25b" event={"ID":"ae86155d-9312-437d-a521-750997c1927e","Type":"ContainerDied","Data":"3fdded661b2dfac39eb2bf2a0ddd926b3539d1da25d674b27206d250e464aa6d"} Jul 03 10:13:58.962720 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.962376 2573 scope.go:117] "RemoveContainer" containerID="48c412fac93eae7ae5cfab898b9039908cc949b85fc72a87f60f9f2c2b034c94" Jul 03 10:13:58.970581 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.970553 2573 scope.go:117] "RemoveContainer" containerID="48c412fac93eae7ae5cfab898b9039908cc949b85fc72a87f60f9f2c2b034c94" Jul 03 10:13:58.970943 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:13:58.970924 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48c412fac93eae7ae5cfab898b9039908cc949b85fc72a87f60f9f2c2b034c94\": container with ID starting with 48c412fac93eae7ae5cfab898b9039908cc949b85fc72a87f60f9f2c2b034c94 not found: ID does not exist" containerID="48c412fac93eae7ae5cfab898b9039908cc949b85fc72a87f60f9f2c2b034c94" Jul 03 10:13:58.971005 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.970951 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48c412fac93eae7ae5cfab898b9039908cc949b85fc72a87f60f9f2c2b034c94"} err="failed to get container status \"48c412fac93eae7ae5cfab898b9039908cc949b85fc72a87f60f9f2c2b034c94\": rpc error: code = NotFound desc = could not find container \"48c412fac93eae7ae5cfab898b9039908cc949b85fc72a87f60f9f2c2b034c94\": container with ID starting with 48c412fac93eae7ae5cfab898b9039908cc949b85fc72a87f60f9f2c2b034c94 not found: ID does not exist" Jul 03 10:13:58.990762 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.990724 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8496857844-lm25b"] Jul 03 10:13:58.998712 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:13:58.998692 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8496857844-lm25b"] Jul 03 10:14:00.562712 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:00.562678 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae86155d-9312-437d-a521-750997c1927e" path="/var/lib/kubelet/pods/ae86155d-9312-437d-a521-750997c1927e/volumes" Jul 03 10:14:23.074626 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:23.074596 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zd77x"] Jul 03 10:14:23.075035 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:23.074947 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae86155d-9312-437d-a521-750997c1927e" containerName="console" Jul 03 10:14:23.075035 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:23.074959 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae86155d-9312-437d-a521-750997c1927e" containerName="console" Jul 03 10:14:23.075035 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:23.075032 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae86155d-9312-437d-a521-750997c1927e" containerName="console" Jul 03 10:14:23.078256 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:23.078241 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zd77x" Jul 03 10:14:23.081549 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:23.081530 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 03 10:14:23.081892 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:23.081875 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-snd9l\"" Jul 03 10:14:23.099840 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:23.099805 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zd77x"] Jul 03 10:14:23.154814 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:23.154780 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zd77x"] Jul 03 10:14:23.186497 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:23.186470 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgtst\" (UniqueName: \"kubernetes.io/projected/0730a6a0-b0f9-4a22-9f11-7b850d194116-kube-api-access-pgtst\") pod \"limitador-limitador-64c94fd64f-zd77x\" (UID: \"0730a6a0-b0f9-4a22-9f11-7b850d194116\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zd77x" Jul 03 10:14:23.186641 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:23.186509 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0730a6a0-b0f9-4a22-9f11-7b850d194116-config-file\") pod \"limitador-limitador-64c94fd64f-zd77x\" (UID: \"0730a6a0-b0f9-4a22-9f11-7b850d194116\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zd77x" Jul 03 10:14:23.287573 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:23.287532 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0730a6a0-b0f9-4a22-9f11-7b850d194116-config-file\") pod \"limitador-limitador-64c94fd64f-zd77x\" (UID: \"0730a6a0-b0f9-4a22-9f11-7b850d194116\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zd77x" Jul 03 10:14:23.287810 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:23.287688 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgtst\" (UniqueName: \"kubernetes.io/projected/0730a6a0-b0f9-4a22-9f11-7b850d194116-kube-api-access-pgtst\") pod \"limitador-limitador-64c94fd64f-zd77x\" (UID: \"0730a6a0-b0f9-4a22-9f11-7b850d194116\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zd77x" Jul 03 10:14:23.288186 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:23.288167 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0730a6a0-b0f9-4a22-9f11-7b850d194116-config-file\") pod \"limitador-limitador-64c94fd64f-zd77x\" (UID: \"0730a6a0-b0f9-4a22-9f11-7b850d194116\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zd77x" Jul 03 10:14:23.301676 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:23.301640 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgtst\" (UniqueName: \"kubernetes.io/projected/0730a6a0-b0f9-4a22-9f11-7b850d194116-kube-api-access-pgtst\") pod \"limitador-limitador-64c94fd64f-zd77x\" (UID: \"0730a6a0-b0f9-4a22-9f11-7b850d194116\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zd77x" Jul 03 10:14:23.388259 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:23.388232 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zd77x" Jul 03 10:14:23.528258 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:23.528222 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zd77x"] Jul 03 10:14:23.531626 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:14:23.531598 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0730a6a0_b0f9_4a22_9f11_7b850d194116.slice/crio-37381708c96148ed07d8c34594633142b6c5e5ccdc06a2bbe5787753df61f7fa WatchSource:0}: Error finding container 37381708c96148ed07d8c34594633142b6c5e5ccdc06a2bbe5787753df61f7fa: Status 404 returned error can't find the container with id 37381708c96148ed07d8c34594633142b6c5e5ccdc06a2bbe5787753df61f7fa Jul 03 10:14:24.081186 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:24.081150 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zd77x" event={"ID":"0730a6a0-b0f9-4a22-9f11-7b850d194116","Type":"ContainerStarted","Data":"37381708c96148ed07d8c34594633142b6c5e5ccdc06a2bbe5787753df61f7fa"} Jul 03 10:14:24.082062 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:24.082043 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-477bf"] Jul 03 10:14:24.086609 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:24.086591 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-477bf" Jul 03 10:14:24.089827 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:24.089808 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-w6vxx\"" Jul 03 10:14:24.094246 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:24.093949 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-477bf"] Jul 03 10:14:24.195470 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:24.195436 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9j77\" (UniqueName: \"kubernetes.io/projected/493ec9c6-0622-42df-ab65-888cf24f1197-kube-api-access-w9j77\") pod \"authorino-75d556bd77-477bf\" (UID: \"493ec9c6-0622-42df-ab65-888cf24f1197\") " pod="kuadrant-system/authorino-75d556bd77-477bf" Jul 03 10:14:24.296526 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:24.296406 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9j77\" (UniqueName: \"kubernetes.io/projected/493ec9c6-0622-42df-ab65-888cf24f1197-kube-api-access-w9j77\") pod \"authorino-75d556bd77-477bf\" (UID: \"493ec9c6-0622-42df-ab65-888cf24f1197\") " pod="kuadrant-system/authorino-75d556bd77-477bf" Jul 03 10:14:24.307927 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:24.307869 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9j77\" (UniqueName: \"kubernetes.io/projected/493ec9c6-0622-42df-ab65-888cf24f1197-kube-api-access-w9j77\") pod \"authorino-75d556bd77-477bf\" (UID: \"493ec9c6-0622-42df-ab65-888cf24f1197\") " pod="kuadrant-system/authorino-75d556bd77-477bf" Jul 03 10:14:24.397287 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:24.397017 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-477bf" Jul 03 10:14:24.569043 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:24.569010 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-477bf"] Jul 03 10:14:24.576193 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:14:24.576057 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod493ec9c6_0622_42df_ab65_888cf24f1197.slice/crio-2a62d01effb8e317ae7a23cce01bd1dffccc17743ef6ed4cc46fcdcbed5dce44 WatchSource:0}: Error finding container 2a62d01effb8e317ae7a23cce01bd1dffccc17743ef6ed4cc46fcdcbed5dce44: Status 404 returned error can't find the container with id 2a62d01effb8e317ae7a23cce01bd1dffccc17743ef6ed4cc46fcdcbed5dce44 Jul 03 10:14:25.091634 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:25.091601 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-477bf" event={"ID":"493ec9c6-0622-42df-ab65-888cf24f1197","Type":"ContainerStarted","Data":"2a62d01effb8e317ae7a23cce01bd1dffccc17743ef6ed4cc46fcdcbed5dce44"} Jul 03 10:14:27.105593 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:27.105556 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zd77x" event={"ID":"0730a6a0-b0f9-4a22-9f11-7b850d194116","Type":"ContainerStarted","Data":"d4cbca3a1dc4f26fd2cb322716265a075ae4a748bd56fff37e9e75470a1ff88e"} Jul 03 10:14:27.106016 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:27.105703 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-zd77x" Jul 03 10:14:27.125792 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:27.125722 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-zd77x" podStartSLOduration=1.217171272 podStartE2EDuration="4.125708303s" totalImagesPullingTime="2.908537031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:14:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:14:27.12426546 +0000 UTC m=+522.036154765" watchObservedRunningTime="2026-07-03 10:14:27.125708303 +0000 UTC m=+522.037597608" Jul 03 10:14:29.117253 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:29.117216 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-477bf" event={"ID":"493ec9c6-0622-42df-ab65-888cf24f1197","Type":"ContainerStarted","Data":"03df2d7ca4fcf75c55cb6be65237167d0e905e56b39bac36b2c65024e28611e9"} Jul 03 10:14:29.135222 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:29.135171 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-477bf" podStartSLOduration=1.241746057 podStartE2EDuration="5.135156759s" totalImagesPullingTime="3.893410702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:14:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:14:29.134368451 +0000 UTC m=+524.046257755" watchObservedRunningTime="2026-07-03 10:14:29.135156759 +0000 UTC m=+524.047046062" Jul 03 10:14:38.112940 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:38.112856 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-zd77x" Jul 03 10:14:40.754130 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:40.754097 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zd77x"] Jul 03 10:14:40.754604 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:40.754317 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-zd77x" podUID="0730a6a0-b0f9-4a22-9f11-7b850d194116" containerName="limitador" containerID="cri-o://d4cbca3a1dc4f26fd2cb322716265a075ae4a748bd56fff37e9e75470a1ff88e" gracePeriod=30 Jul 03 10:14:41.174844 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:41.174811 2573 generic.go:356] "Generic (PLEG): container finished" podID="0730a6a0-b0f9-4a22-9f11-7b850d194116" containerID="d4cbca3a1dc4f26fd2cb322716265a075ae4a748bd56fff37e9e75470a1ff88e" exitCode=0 Jul 03 10:14:41.174996 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:41.174891 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zd77x" event={"ID":"0730a6a0-b0f9-4a22-9f11-7b850d194116","Type":"ContainerDied","Data":"d4cbca3a1dc4f26fd2cb322716265a075ae4a748bd56fff37e9e75470a1ff88e"} Jul 03 10:14:41.293181 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:41.293156 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zd77x" Jul 03 10:14:41.360461 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:41.360380 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgtst\" (UniqueName: \"kubernetes.io/projected/0730a6a0-b0f9-4a22-9f11-7b850d194116-kube-api-access-pgtst\") pod \"0730a6a0-b0f9-4a22-9f11-7b850d194116\" (UID: \"0730a6a0-b0f9-4a22-9f11-7b850d194116\") " Jul 03 10:14:41.360461 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:41.360428 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0730a6a0-b0f9-4a22-9f11-7b850d194116-config-file\") pod \"0730a6a0-b0f9-4a22-9f11-7b850d194116\" (UID: \"0730a6a0-b0f9-4a22-9f11-7b850d194116\") " Jul 03 10:14:41.360840 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:41.360813 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0730a6a0-b0f9-4a22-9f11-7b850d194116-config-file" (OuterVolumeSpecName: "config-file") pod "0730a6a0-b0f9-4a22-9f11-7b850d194116" (UID: "0730a6a0-b0f9-4a22-9f11-7b850d194116"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:14:41.362545 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:41.362525 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0730a6a0-b0f9-4a22-9f11-7b850d194116-kube-api-access-pgtst" (OuterVolumeSpecName: "kube-api-access-pgtst") pod "0730a6a0-b0f9-4a22-9f11-7b850d194116" (UID: "0730a6a0-b0f9-4a22-9f11-7b850d194116"). InnerVolumeSpecName "kube-api-access-pgtst". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:14:41.461282 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:41.461244 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgtst\" (UniqueName: \"kubernetes.io/projected/0730a6a0-b0f9-4a22-9f11-7b850d194116-kube-api-access-pgtst\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:14:41.461282 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:41.461276 2573 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0730a6a0-b0f9-4a22-9f11-7b850d194116-config-file\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:14:42.180919 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:42.180893 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zd77x" Jul 03 10:14:42.181368 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:42.180918 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zd77x" event={"ID":"0730a6a0-b0f9-4a22-9f11-7b850d194116","Type":"ContainerDied","Data":"37381708c96148ed07d8c34594633142b6c5e5ccdc06a2bbe5787753df61f7fa"} Jul 03 10:14:42.181368 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:42.180971 2573 scope.go:117] "RemoveContainer" containerID="d4cbca3a1dc4f26fd2cb322716265a075ae4a748bd56fff37e9e75470a1ff88e" Jul 03 10:14:42.203330 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:42.203300 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zd77x"] Jul 03 10:14:42.207602 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:42.207580 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zd77x"] Jul 03 10:14:42.562437 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:42.562351 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0730a6a0-b0f9-4a22-9f11-7b850d194116" path="/var/lib/kubelet/pods/0730a6a0-b0f9-4a22-9f11-7b850d194116/volumes" Jul 03 10:14:44.588493 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:44.588459 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-dnsb8"] Jul 03 10:14:44.589081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:44.589061 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0730a6a0-b0f9-4a22-9f11-7b850d194116" containerName="limitador" Jul 03 10:14:44.589140 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:44.589085 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0730a6a0-b0f9-4a22-9f11-7b850d194116" containerName="limitador" Jul 03 10:14:44.589219 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:44.589207 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0730a6a0-b0f9-4a22-9f11-7b850d194116" containerName="limitador" Jul 03 10:14:44.593792 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:44.593768 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-dnsb8" Jul 03 10:14:44.597035 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:44.597012 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 03 10:14:44.597225 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:44.597014 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-gxjz5\"" Jul 03 10:14:44.601260 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:44.601237 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-dnsb8"] Jul 03 10:14:44.683621 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:44.683589 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/65ea69a1-53b0-42b0-a6fd-197ef6bc23f4-data\") pod \"postgres-69fb4594cf-dnsb8\" (UID: \"65ea69a1-53b0-42b0-a6fd-197ef6bc23f4\") " pod="opendatahub/postgres-69fb4594cf-dnsb8" Jul 03 10:14:44.683810 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:44.683643 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rv58\" (UniqueName: \"kubernetes.io/projected/65ea69a1-53b0-42b0-a6fd-197ef6bc23f4-kube-api-access-6rv58\") pod \"postgres-69fb4594cf-dnsb8\" (UID: \"65ea69a1-53b0-42b0-a6fd-197ef6bc23f4\") " pod="opendatahub/postgres-69fb4594cf-dnsb8" Jul 03 10:14:44.784407 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:44.784365 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/65ea69a1-53b0-42b0-a6fd-197ef6bc23f4-data\") pod \"postgres-69fb4594cf-dnsb8\" (UID: \"65ea69a1-53b0-42b0-a6fd-197ef6bc23f4\") " pod="opendatahub/postgres-69fb4594cf-dnsb8" Jul 03 10:14:44.784579 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:44.784430 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rv58\" (UniqueName: \"kubernetes.io/projected/65ea69a1-53b0-42b0-a6fd-197ef6bc23f4-kube-api-access-6rv58\") pod \"postgres-69fb4594cf-dnsb8\" (UID: \"65ea69a1-53b0-42b0-a6fd-197ef6bc23f4\") " pod="opendatahub/postgres-69fb4594cf-dnsb8" Jul 03 10:14:44.784796 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:44.784776 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/65ea69a1-53b0-42b0-a6fd-197ef6bc23f4-data\") pod \"postgres-69fb4594cf-dnsb8\" (UID: \"65ea69a1-53b0-42b0-a6fd-197ef6bc23f4\") " pod="opendatahub/postgres-69fb4594cf-dnsb8" Jul 03 10:14:44.794127 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:44.794104 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rv58\" (UniqueName: \"kubernetes.io/projected/65ea69a1-53b0-42b0-a6fd-197ef6bc23f4-kube-api-access-6rv58\") pod \"postgres-69fb4594cf-dnsb8\" (UID: \"65ea69a1-53b0-42b0-a6fd-197ef6bc23f4\") " pod="opendatahub/postgres-69fb4594cf-dnsb8" Jul 03 10:14:44.905117 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:44.905089 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-dnsb8" Jul 03 10:14:45.042847 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:45.042819 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-dnsb8"] Jul 03 10:14:45.044912 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:14:45.044871 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65ea69a1_53b0_42b0_a6fd_197ef6bc23f4.slice/crio-0fbea06c9d30992210c5caac188800a92df3fd3e072312f260c91a4c3fb2b671 WatchSource:0}: Error finding container 0fbea06c9d30992210c5caac188800a92df3fd3e072312f260c91a4c3fb2b671: Status 404 returned error can't find the container with id 0fbea06c9d30992210c5caac188800a92df3fd3e072312f260c91a4c3fb2b671 Jul 03 10:14:45.196364 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:45.196283 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-dnsb8" event={"ID":"65ea69a1-53b0-42b0-a6fd-197ef6bc23f4","Type":"ContainerStarted","Data":"0fbea06c9d30992210c5caac188800a92df3fd3e072312f260c91a4c3fb2b671"} Jul 03 10:14:50.313153 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:50.313130 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 03 10:14:51.229178 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:51.229142 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-dnsb8" event={"ID":"65ea69a1-53b0-42b0-a6fd-197ef6bc23f4","Type":"ContainerStarted","Data":"372e5b5c698f3ef5216edbd683b70e2eb66074ae25582999395b75d4a882546c"} Jul 03 10:14:51.229356 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:51.229264 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-dnsb8" Jul 03 10:14:51.248853 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:51.248801 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-dnsb8" podStartSLOduration=1.984950964 podStartE2EDuration="7.248786766s" totalImagesPullingTime="5.263835802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:14:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:14:51.247612485 +0000 UTC m=+546.159501791" watchObservedRunningTime="2026-07-03 10:14:51.248786766 +0000 UTC m=+546.160676070" Jul 03 10:14:57.263059 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:57.263027 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-dnsb8" Jul 03 10:14:57.826335 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:57.826301 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qvwvp"] Jul 03 10:14:57.831533 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:57.831512 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-qvwvp" Jul 03 10:14:57.839674 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:57.839650 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qvwvp"] Jul 03 10:14:57.903120 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:57.903082 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q45qj\" (UniqueName: \"kubernetes.io/projected/3b2ce07f-d0d0-4fa3-8317-8108d7b138e3-kube-api-access-q45qj\") pod \"authorino-5d4b67c446-qvwvp\" (UID: \"3b2ce07f-d0d0-4fa3-8317-8108d7b138e3\") " pod="kuadrant-system/authorino-5d4b67c446-qvwvp" Jul 03 10:14:58.004436 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.004399 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q45qj\" (UniqueName: \"kubernetes.io/projected/3b2ce07f-d0d0-4fa3-8317-8108d7b138e3-kube-api-access-q45qj\") pod \"authorino-5d4b67c446-qvwvp\" (UID: \"3b2ce07f-d0d0-4fa3-8317-8108d7b138e3\") " pod="kuadrant-system/authorino-5d4b67c446-qvwvp" Jul 03 10:14:58.017727 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.017688 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q45qj\" (UniqueName: \"kubernetes.io/projected/3b2ce07f-d0d0-4fa3-8317-8108d7b138e3-kube-api-access-q45qj\") pod \"authorino-5d4b67c446-qvwvp\" (UID: \"3b2ce07f-d0d0-4fa3-8317-8108d7b138e3\") " pod="kuadrant-system/authorino-5d4b67c446-qvwvp" Jul 03 10:14:58.096661 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.096559 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qvwvp"] Jul 03 10:14:58.096915 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.096897 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-qvwvp" Jul 03 10:14:58.134353 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.134319 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6655f5998d-tpl4l"] Jul 03 10:14:58.139545 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.139519 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6655f5998d-tpl4l" Jul 03 10:14:58.145143 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.145116 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6655f5998d-tpl4l"] Jul 03 10:14:58.206207 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.206178 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkr48\" (UniqueName: \"kubernetes.io/projected/3b9d6b43-974f-4247-bd21-cdd6f0f3921e-kube-api-access-lkr48\") pod \"authorino-6655f5998d-tpl4l\" (UID: \"3b9d6b43-974f-4247-bd21-cdd6f0f3921e\") " pod="kuadrant-system/authorino-6655f5998d-tpl4l" Jul 03 10:14:58.231021 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.230985 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qvwvp"] Jul 03 10:14:58.236295 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:14:58.236265 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b2ce07f_d0d0_4fa3_8317_8108d7b138e3.slice/crio-528a4ba721c5331286ce514f9ffc03aa9170bd9957427bd77c0d78fddbfff8a6 WatchSource:0}: Error finding container 528a4ba721c5331286ce514f9ffc03aa9170bd9957427bd77c0d78fddbfff8a6: Status 404 returned error can't find the container with id 528a4ba721c5331286ce514f9ffc03aa9170bd9957427bd77c0d78fddbfff8a6 Jul 03 10:14:58.262121 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.262093 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-qvwvp" event={"ID":"3b2ce07f-d0d0-4fa3-8317-8108d7b138e3","Type":"ContainerStarted","Data":"528a4ba721c5331286ce514f9ffc03aa9170bd9957427bd77c0d78fddbfff8a6"} Jul 03 10:14:58.306858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.306829 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkr48\" (UniqueName: \"kubernetes.io/projected/3b9d6b43-974f-4247-bd21-cdd6f0f3921e-kube-api-access-lkr48\") pod \"authorino-6655f5998d-tpl4l\" (UID: \"3b9d6b43-974f-4247-bd21-cdd6f0f3921e\") " pod="kuadrant-system/authorino-6655f5998d-tpl4l" Jul 03 10:14:58.319692 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.319654 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6655f5998d-tpl4l"] Jul 03 10:14:58.320017 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:14:58.319995 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-lkr48], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6655f5998d-tpl4l" podUID="3b9d6b43-974f-4247-bd21-cdd6f0f3921e" Jul 03 10:14:58.325491 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.325466 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkr48\" (UniqueName: \"kubernetes.io/projected/3b9d6b43-974f-4247-bd21-cdd6f0f3921e-kube-api-access-lkr48\") pod \"authorino-6655f5998d-tpl4l\" (UID: \"3b9d6b43-974f-4247-bd21-cdd6f0f3921e\") " pod="kuadrant-system/authorino-6655f5998d-tpl4l" Jul 03 10:14:58.341353 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.341325 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6b6d65c7c6-f479c"] Jul 03 10:14:58.344948 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.344932 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b6d65c7c6-f479c" Jul 03 10:14:58.348258 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.348205 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 03 10:14:58.355273 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.355234 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b6d65c7c6-f479c"] Jul 03 10:14:58.407670 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.407637 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/59a6aab5-e01f-4f27-b2e4-665842fe7bfd-tls-cert\") pod \"authorino-6b6d65c7c6-f479c\" (UID: \"59a6aab5-e01f-4f27-b2e4-665842fe7bfd\") " pod="kuadrant-system/authorino-6b6d65c7c6-f479c" Jul 03 10:14:58.407869 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.407682 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59s89\" (UniqueName: \"kubernetes.io/projected/59a6aab5-e01f-4f27-b2e4-665842fe7bfd-kube-api-access-59s89\") pod \"authorino-6b6d65c7c6-f479c\" (UID: \"59a6aab5-e01f-4f27-b2e4-665842fe7bfd\") " pod="kuadrant-system/authorino-6b6d65c7c6-f479c" Jul 03 10:14:58.509146 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.509099 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/59a6aab5-e01f-4f27-b2e4-665842fe7bfd-tls-cert\") pod \"authorino-6b6d65c7c6-f479c\" (UID: \"59a6aab5-e01f-4f27-b2e4-665842fe7bfd\") " pod="kuadrant-system/authorino-6b6d65c7c6-f479c" Jul 03 10:14:58.509377 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.509163 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59s89\" (UniqueName: \"kubernetes.io/projected/59a6aab5-e01f-4f27-b2e4-665842fe7bfd-kube-api-access-59s89\") pod \"authorino-6b6d65c7c6-f479c\" (UID: \"59a6aab5-e01f-4f27-b2e4-665842fe7bfd\") " pod="kuadrant-system/authorino-6b6d65c7c6-f479c" Jul 03 10:14:58.511481 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.511460 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/59a6aab5-e01f-4f27-b2e4-665842fe7bfd-tls-cert\") pod \"authorino-6b6d65c7c6-f479c\" (UID: \"59a6aab5-e01f-4f27-b2e4-665842fe7bfd\") " pod="kuadrant-system/authorino-6b6d65c7c6-f479c" Jul 03 10:14:58.517374 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.517355 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59s89\" (UniqueName: \"kubernetes.io/projected/59a6aab5-e01f-4f27-b2e4-665842fe7bfd-kube-api-access-59s89\") pod \"authorino-6b6d65c7c6-f479c\" (UID: \"59a6aab5-e01f-4f27-b2e4-665842fe7bfd\") " pod="kuadrant-system/authorino-6b6d65c7c6-f479c" Jul 03 10:14:58.654666 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.654645 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b6d65c7c6-f479c" Jul 03 10:14:58.793654 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:58.793617 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b6d65c7c6-f479c"] Jul 03 10:14:58.797047 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:14:58.797017 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59a6aab5_e01f_4f27_b2e4_665842fe7bfd.slice/crio-21586822192e1cb06c16a7e43f49715b94cb2f00f4ce832d89fcda1859d97b0c WatchSource:0}: Error finding container 21586822192e1cb06c16a7e43f49715b94cb2f00f4ce832d89fcda1859d97b0c: Status 404 returned error can't find the container with id 21586822192e1cb06c16a7e43f49715b94cb2f00f4ce832d89fcda1859d97b0c Jul 03 10:14:59.269390 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.269344 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b6d65c7c6-f479c" event={"ID":"59a6aab5-e01f-4f27-b2e4-665842fe7bfd","Type":"ContainerStarted","Data":"2e52d8d6c37ed9321f4c2600acdd3e1e8ca67ac85a2588f30927bc541df4837c"} Jul 03 10:14:59.269578 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.269403 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b6d65c7c6-f479c" event={"ID":"59a6aab5-e01f-4f27-b2e4-665842fe7bfd","Type":"ContainerStarted","Data":"21586822192e1cb06c16a7e43f49715b94cb2f00f4ce832d89fcda1859d97b0c"} Jul 03 10:14:59.271339 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.271308 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6655f5998d-tpl4l" Jul 03 10:14:59.271696 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.271653 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-qvwvp" podUID="3b2ce07f-d0d0-4fa3-8317-8108d7b138e3" containerName="authorino" containerID="cri-o://9da0dc4493c15e0add73b1a77304ac647a8e2f9e05ad1d737eee086c16d4df1e" gracePeriod=30 Jul 03 10:14:59.271837 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.271798 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-qvwvp" event={"ID":"3b2ce07f-d0d0-4fa3-8317-8108d7b138e3","Type":"ContainerStarted","Data":"9da0dc4493c15e0add73b1a77304ac647a8e2f9e05ad1d737eee086c16d4df1e"} Jul 03 10:14:59.279149 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.279130 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6655f5998d-tpl4l" Jul 03 10:14:59.287551 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.287494 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6b6d65c7c6-f479c" podStartSLOduration=0.908616225 podStartE2EDuration="1.287474516s" totalImagesPullingTime="378.858291ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:14:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:14:59.286263023 +0000 UTC m=+554.198152328" watchObservedRunningTime="2026-07-03 10:14:59.287474516 +0000 UTC m=+554.199363818" Jul 03 10:14:59.304412 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.304349 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-qvwvp" podStartSLOduration=1.8874431999999999 podStartE2EDuration="2.304327845s" totalImagesPullingTime="416.884645ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:14:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:14:59.302958078 +0000 UTC m=+554.214847385" watchObservedRunningTime="2026-07-03 10:14:59.304327845 +0000 UTC m=+554.216217151" Jul 03 10:14:59.315404 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.315346 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkr48\" (UniqueName: \"kubernetes.io/projected/3b9d6b43-974f-4247-bd21-cdd6f0f3921e-kube-api-access-lkr48\") pod \"3b9d6b43-974f-4247-bd21-cdd6f0f3921e\" (UID: \"3b9d6b43-974f-4247-bd21-cdd6f0f3921e\") " Jul 03 10:14:59.317236 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.317206 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-477bf"] Jul 03 10:14:59.317459 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.317419 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-477bf" podUID="493ec9c6-0622-42df-ab65-888cf24f1197" containerName="authorino" containerID="cri-o://03df2d7ca4fcf75c55cb6be65237167d0e905e56b39bac36b2c65024e28611e9" gracePeriod=30 Jul 03 10:14:59.318623 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.318593 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b9d6b43-974f-4247-bd21-cdd6f0f3921e-kube-api-access-lkr48" (OuterVolumeSpecName: "kube-api-access-lkr48") pod "3b9d6b43-974f-4247-bd21-cdd6f0f3921e" (UID: "3b9d6b43-974f-4247-bd21-cdd6f0f3921e"). InnerVolumeSpecName "kube-api-access-lkr48". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:14:59.416719 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.416681 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkr48\" (UniqueName: \"kubernetes.io/projected/3b9d6b43-974f-4247-bd21-cdd6f0f3921e-kube-api-access-lkr48\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:14:59.549702 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.549672 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-qvwvp" Jul 03 10:14:59.586473 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.586447 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-477bf" Jul 03 10:14:59.618091 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.618059 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9j77\" (UniqueName: \"kubernetes.io/projected/493ec9c6-0622-42df-ab65-888cf24f1197-kube-api-access-w9j77\") pod \"493ec9c6-0622-42df-ab65-888cf24f1197\" (UID: \"493ec9c6-0622-42df-ab65-888cf24f1197\") " Jul 03 10:14:59.618267 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.618131 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q45qj\" (UniqueName: \"kubernetes.io/projected/3b2ce07f-d0d0-4fa3-8317-8108d7b138e3-kube-api-access-q45qj\") pod \"3b2ce07f-d0d0-4fa3-8317-8108d7b138e3\" (UID: \"3b2ce07f-d0d0-4fa3-8317-8108d7b138e3\") " Jul 03 10:14:59.620584 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.620540 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b2ce07f-d0d0-4fa3-8317-8108d7b138e3-kube-api-access-q45qj" (OuterVolumeSpecName: "kube-api-access-q45qj") pod "3b2ce07f-d0d0-4fa3-8317-8108d7b138e3" (UID: "3b2ce07f-d0d0-4fa3-8317-8108d7b138e3"). InnerVolumeSpecName "kube-api-access-q45qj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:14:59.620584 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.620555 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/493ec9c6-0622-42df-ab65-888cf24f1197-kube-api-access-w9j77" (OuterVolumeSpecName: "kube-api-access-w9j77") pod "493ec9c6-0622-42df-ab65-888cf24f1197" (UID: "493ec9c6-0622-42df-ab65-888cf24f1197"). InnerVolumeSpecName "kube-api-access-w9j77". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:14:59.719128 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.719042 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9j77\" (UniqueName: \"kubernetes.io/projected/493ec9c6-0622-42df-ab65-888cf24f1197-kube-api-access-w9j77\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:14:59.719128 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:14:59.719072 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q45qj\" (UniqueName: \"kubernetes.io/projected/3b2ce07f-d0d0-4fa3-8317-8108d7b138e3-kube-api-access-q45qj\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:15:00.277146 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.277067 2573 generic.go:356] "Generic (PLEG): container finished" podID="3b2ce07f-d0d0-4fa3-8317-8108d7b138e3" containerID="9da0dc4493c15e0add73b1a77304ac647a8e2f9e05ad1d737eee086c16d4df1e" exitCode=0 Jul 03 10:15:00.277146 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.277107 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-qvwvp" Jul 03 10:15:00.277374 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.277158 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-qvwvp" event={"ID":"3b2ce07f-d0d0-4fa3-8317-8108d7b138e3","Type":"ContainerDied","Data":"9da0dc4493c15e0add73b1a77304ac647a8e2f9e05ad1d737eee086c16d4df1e"} Jul 03 10:15:00.277374 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.277202 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-qvwvp" event={"ID":"3b2ce07f-d0d0-4fa3-8317-8108d7b138e3","Type":"ContainerDied","Data":"528a4ba721c5331286ce514f9ffc03aa9170bd9957427bd77c0d78fddbfff8a6"} Jul 03 10:15:00.277374 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.277219 2573 scope.go:117] "RemoveContainer" containerID="9da0dc4493c15e0add73b1a77304ac647a8e2f9e05ad1d737eee086c16d4df1e" Jul 03 10:15:00.278640 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.278388 2573 generic.go:356] "Generic (PLEG): container finished" podID="493ec9c6-0622-42df-ab65-888cf24f1197" containerID="03df2d7ca4fcf75c55cb6be65237167d0e905e56b39bac36b2c65024e28611e9" exitCode=0 Jul 03 10:15:00.278640 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.278410 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-477bf" event={"ID":"493ec9c6-0622-42df-ab65-888cf24f1197","Type":"ContainerDied","Data":"03df2d7ca4fcf75c55cb6be65237167d0e905e56b39bac36b2c65024e28611e9"} Jul 03 10:15:00.278640 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.278433 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-477bf" Jul 03 10:15:00.278640 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.278440 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-477bf" event={"ID":"493ec9c6-0622-42df-ab65-888cf24f1197","Type":"ContainerDied","Data":"2a62d01effb8e317ae7a23cce01bd1dffccc17743ef6ed4cc46fcdcbed5dce44"} Jul 03 10:15:00.278640 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.278525 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6655f5998d-tpl4l" Jul 03 10:15:00.286306 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.286284 2573 scope.go:117] "RemoveContainer" containerID="9da0dc4493c15e0add73b1a77304ac647a8e2f9e05ad1d737eee086c16d4df1e" Jul 03 10:15:00.286555 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:15:00.286534 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9da0dc4493c15e0add73b1a77304ac647a8e2f9e05ad1d737eee086c16d4df1e\": container with ID starting with 9da0dc4493c15e0add73b1a77304ac647a8e2f9e05ad1d737eee086c16d4df1e not found: ID does not exist" containerID="9da0dc4493c15e0add73b1a77304ac647a8e2f9e05ad1d737eee086c16d4df1e" Jul 03 10:15:00.286638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.286567 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9da0dc4493c15e0add73b1a77304ac647a8e2f9e05ad1d737eee086c16d4df1e"} err="failed to get container status \"9da0dc4493c15e0add73b1a77304ac647a8e2f9e05ad1d737eee086c16d4df1e\": rpc error: code = NotFound desc = could not find container \"9da0dc4493c15e0add73b1a77304ac647a8e2f9e05ad1d737eee086c16d4df1e\": container with ID starting with 9da0dc4493c15e0add73b1a77304ac647a8e2f9e05ad1d737eee086c16d4df1e not found: ID does not exist" Jul 03 10:15:00.286638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.286590 2573 scope.go:117] "RemoveContainer" containerID="03df2d7ca4fcf75c55cb6be65237167d0e905e56b39bac36b2c65024e28611e9" Jul 03 10:15:00.295049 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.295027 2573 scope.go:117] "RemoveContainer" containerID="03df2d7ca4fcf75c55cb6be65237167d0e905e56b39bac36b2c65024e28611e9" Jul 03 10:15:00.295290 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:15:00.295273 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03df2d7ca4fcf75c55cb6be65237167d0e905e56b39bac36b2c65024e28611e9\": container with ID starting with 03df2d7ca4fcf75c55cb6be65237167d0e905e56b39bac36b2c65024e28611e9 not found: ID does not exist" containerID="03df2d7ca4fcf75c55cb6be65237167d0e905e56b39bac36b2c65024e28611e9" Jul 03 10:15:00.295340 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.295296 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03df2d7ca4fcf75c55cb6be65237167d0e905e56b39bac36b2c65024e28611e9"} err="failed to get container status \"03df2d7ca4fcf75c55cb6be65237167d0e905e56b39bac36b2c65024e28611e9\": rpc error: code = NotFound desc = could not find container \"03df2d7ca4fcf75c55cb6be65237167d0e905e56b39bac36b2c65024e28611e9\": container with ID starting with 03df2d7ca4fcf75c55cb6be65237167d0e905e56b39bac36b2c65024e28611e9 not found: ID does not exist" Jul 03 10:15:00.321138 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.321104 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6655f5998d-tpl4l"] Jul 03 10:15:00.324383 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.324359 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6655f5998d-tpl4l"] Jul 03 10:15:00.336793 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.336765 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qvwvp"] Jul 03 10:15:00.341772 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.341751 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-qvwvp"] Jul 03 10:15:00.353584 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.353553 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-477bf"] Jul 03 10:15:00.355995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.355976 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-477bf"] Jul 03 10:15:00.562967 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.562887 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b2ce07f-d0d0-4fa3-8317-8108d7b138e3" path="/var/lib/kubelet/pods/3b2ce07f-d0d0-4fa3-8317-8108d7b138e3/volumes" Jul 03 10:15:00.563209 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.563196 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b9d6b43-974f-4247-bd21-cdd6f0f3921e" path="/var/lib/kubelet/pods/3b9d6b43-974f-4247-bd21-cdd6f0f3921e/volumes" Jul 03 10:15:00.563395 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:00.563384 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="493ec9c6-0622-42df-ab65-888cf24f1197" path="/var/lib/kubelet/pods/493ec9c6-0622-42df-ab65-888cf24f1197/volumes" Jul 03 10:15:42.202553 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.202521 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-9c9cx"] Jul 03 10:15:42.203070 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.202987 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b2ce07f-d0d0-4fa3-8317-8108d7b138e3" containerName="authorino" Jul 03 10:15:42.203070 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.203000 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b2ce07f-d0d0-4fa3-8317-8108d7b138e3" containerName="authorino" Jul 03 10:15:42.203070 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.203019 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="493ec9c6-0622-42df-ab65-888cf24f1197" containerName="authorino" Jul 03 10:15:42.203070 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.203025 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="493ec9c6-0622-42df-ab65-888cf24f1197" containerName="authorino" Jul 03 10:15:42.203202 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.203080 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="493ec9c6-0622-42df-ab65-888cf24f1197" containerName="authorino" Jul 03 10:15:42.203202 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.203091 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b2ce07f-d0d0-4fa3-8317-8108d7b138e3" containerName="authorino" Jul 03 10:15:42.205215 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.205198 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-9c9cx" Jul 03 10:15:42.209830 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.209790 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-594x6\"" Jul 03 10:15:42.210024 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.210005 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 03 10:15:42.210142 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.210125 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-dhnq7"] Jul 03 10:15:42.212878 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.212456 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-dhnq7" Jul 03 10:15:42.217001 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.216976 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-9c9cx"] Jul 03 10:15:42.234280 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.234244 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-dhnq7"] Jul 03 10:15:42.309987 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.309955 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6bd5c00-e546-4115-95eb-802c8cc30ddd-plugins-config-volume\") pod \"payload-processing-58f86c7798-dhnq7\" (UID: \"a6bd5c00-e546-4115-95eb-802c8cc30ddd\") " pod="openshift-ingress/payload-processing-58f86c7798-dhnq7" Jul 03 10:15:42.310173 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.310074 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpbt7\" (UniqueName: \"kubernetes.io/projected/ce5fa191-9b07-47c6-a101-aa1a96f95f07-kube-api-access-rpbt7\") pod \"payload-pre-processing-7964457696-9c9cx\" (UID: \"ce5fa191-9b07-47c6-a101-aa1a96f95f07\") " pod="openshift-ingress/payload-pre-processing-7964457696-9c9cx" Jul 03 10:15:42.310173 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.310103 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce5fa191-9b07-47c6-a101-aa1a96f95f07-plugins-config-volume\") pod \"payload-pre-processing-7964457696-9c9cx\" (UID: \"ce5fa191-9b07-47c6-a101-aa1a96f95f07\") " pod="openshift-ingress/payload-pre-processing-7964457696-9c9cx" Jul 03 10:15:42.310269 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.310197 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kr6j\" (UniqueName: \"kubernetes.io/projected/a6bd5c00-e546-4115-95eb-802c8cc30ddd-kube-api-access-6kr6j\") pod \"payload-processing-58f86c7798-dhnq7\" (UID: \"a6bd5c00-e546-4115-95eb-802c8cc30ddd\") " pod="openshift-ingress/payload-processing-58f86c7798-dhnq7" Jul 03 10:15:42.411280 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.411244 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6bd5c00-e546-4115-95eb-802c8cc30ddd-plugins-config-volume\") pod \"payload-processing-58f86c7798-dhnq7\" (UID: \"a6bd5c00-e546-4115-95eb-802c8cc30ddd\") " pod="openshift-ingress/payload-processing-58f86c7798-dhnq7" Jul 03 10:15:42.411479 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.411320 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpbt7\" (UniqueName: \"kubernetes.io/projected/ce5fa191-9b07-47c6-a101-aa1a96f95f07-kube-api-access-rpbt7\") pod \"payload-pre-processing-7964457696-9c9cx\" (UID: \"ce5fa191-9b07-47c6-a101-aa1a96f95f07\") " pod="openshift-ingress/payload-pre-processing-7964457696-9c9cx" Jul 03 10:15:42.411479 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.411342 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce5fa191-9b07-47c6-a101-aa1a96f95f07-plugins-config-volume\") pod \"payload-pre-processing-7964457696-9c9cx\" (UID: \"ce5fa191-9b07-47c6-a101-aa1a96f95f07\") " pod="openshift-ingress/payload-pre-processing-7964457696-9c9cx" Jul 03 10:15:42.411479 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.411364 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kr6j\" (UniqueName: \"kubernetes.io/projected/a6bd5c00-e546-4115-95eb-802c8cc30ddd-kube-api-access-6kr6j\") pod \"payload-processing-58f86c7798-dhnq7\" (UID: \"a6bd5c00-e546-4115-95eb-802c8cc30ddd\") " pod="openshift-ingress/payload-processing-58f86c7798-dhnq7" Jul 03 10:15:42.412091 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.412066 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce5fa191-9b07-47c6-a101-aa1a96f95f07-plugins-config-volume\") pod \"payload-pre-processing-7964457696-9c9cx\" (UID: \"ce5fa191-9b07-47c6-a101-aa1a96f95f07\") " pod="openshift-ingress/payload-pre-processing-7964457696-9c9cx" Jul 03 10:15:42.412091 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.412077 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6bd5c00-e546-4115-95eb-802c8cc30ddd-plugins-config-volume\") pod \"payload-processing-58f86c7798-dhnq7\" (UID: \"a6bd5c00-e546-4115-95eb-802c8cc30ddd\") " pod="openshift-ingress/payload-processing-58f86c7798-dhnq7" Jul 03 10:15:42.420781 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.420756 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kr6j\" (UniqueName: \"kubernetes.io/projected/a6bd5c00-e546-4115-95eb-802c8cc30ddd-kube-api-access-6kr6j\") pod \"payload-processing-58f86c7798-dhnq7\" (UID: \"a6bd5c00-e546-4115-95eb-802c8cc30ddd\") " pod="openshift-ingress/payload-processing-58f86c7798-dhnq7" Jul 03 10:15:42.420871 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.420764 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpbt7\" (UniqueName: \"kubernetes.io/projected/ce5fa191-9b07-47c6-a101-aa1a96f95f07-kube-api-access-rpbt7\") pod \"payload-pre-processing-7964457696-9c9cx\" (UID: \"ce5fa191-9b07-47c6-a101-aa1a96f95f07\") " pod="openshift-ingress/payload-pre-processing-7964457696-9c9cx" Jul 03 10:15:42.517699 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.517623 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-9c9cx" Jul 03 10:15:42.525435 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.525409 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-dhnq7" Jul 03 10:15:42.687919 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.687892 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-9c9cx"] Jul 03 10:15:42.689993 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:15:42.689951 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce5fa191_9b07_47c6_a101_aa1a96f95f07.slice/crio-a29fb00651b144a0557e469f76d8b61b74ad9d15cd6e6ce6ffdfc5265770dda6 WatchSource:0}: Error finding container a29fb00651b144a0557e469f76d8b61b74ad9d15cd6e6ce6ffdfc5265770dda6: Status 404 returned error can't find the container with id a29fb00651b144a0557e469f76d8b61b74ad9d15cd6e6ce6ffdfc5265770dda6 Jul 03 10:15:42.703616 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:42.703593 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-dhnq7"] Jul 03 10:15:42.705675 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:15:42.705653 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6bd5c00_e546_4115_95eb_802c8cc30ddd.slice/crio-d14fdcec0b67f8bc58e889171365d49a833d99e7677d8172a161248e5d4ac4c0 WatchSource:0}: Error finding container d14fdcec0b67f8bc58e889171365d49a833d99e7677d8172a161248e5d4ac4c0: Status 404 returned error can't find the container with id d14fdcec0b67f8bc58e889171365d49a833d99e7677d8172a161248e5d4ac4c0 Jul 03 10:15:43.488314 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:43.488255 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-dhnq7" event={"ID":"a6bd5c00-e546-4115-95eb-802c8cc30ddd","Type":"ContainerStarted","Data":"d14fdcec0b67f8bc58e889171365d49a833d99e7677d8172a161248e5d4ac4c0"} Jul 03 10:15:43.489889 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:43.489860 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-9c9cx" event={"ID":"ce5fa191-9b07-47c6-a101-aa1a96f95f07","Type":"ContainerStarted","Data":"a29fb00651b144a0557e469f76d8b61b74ad9d15cd6e6ce6ffdfc5265770dda6"} Jul 03 10:15:44.497204 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:44.497167 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-dhnq7" event={"ID":"a6bd5c00-e546-4115-95eb-802c8cc30ddd","Type":"ContainerStarted","Data":"3a4185f1584d68c542e253779b0b296ebf3a97b51ff614d3ed8aa9de71ae99c3"} Jul 03 10:15:44.497676 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:44.497278 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-dhnq7" Jul 03 10:15:44.498633 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:44.498610 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-9c9cx" event={"ID":"ce5fa191-9b07-47c6-a101-aa1a96f95f07","Type":"ContainerStarted","Data":"e54225b49ae077e5db303a930f2ab0a5b9744bc13fe1e4a40cc0e8d721891121"} Jul 03 10:15:44.498759 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:44.498741 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-9c9cx" Jul 03 10:15:44.516718 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:44.516667 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-dhnq7" podStartSLOduration=1.263008784 podStartE2EDuration="2.516646946s" totalImagesPullingTime="1.253638162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:15:44.515830842 +0000 UTC m=+599.427720147" watchObservedRunningTime="2026-07-03 10:15:44.516646946 +0000 UTC m=+599.428536254" Jul 03 10:15:44.537930 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:44.537879 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-9c9cx" podStartSLOduration=1.271306371 podStartE2EDuration="2.537863825s" totalImagesPullingTime="1.266557454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:15:44.535869663 +0000 UTC m=+599.447758968" watchObservedRunningTime="2026-07-03 10:15:44.537863825 +0000 UTC m=+599.449753129" Jul 03 10:15:55.506611 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:55.506580 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-dhnq7" Jul 03 10:15:55.507164 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:55.506918 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-9c9cx" Jul 03 10:15:59.176071 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.176038 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2"] Jul 03 10:15:59.179459 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.179442 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:15:59.183720 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.183696 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5mnsz\"" Jul 03 10:15:59.183720 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.183714 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 03 10:15:59.183909 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.183697 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 03 10:15:59.183909 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.183697 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 03 10:15:59.190230 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.190209 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2"] Jul 03 10:15:59.271512 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.271479 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f6e1ffc-2fb8-4fcc-9938-097e44e6863b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2\" (UID: \"6f6e1ffc-2fb8-4fcc-9938-097e44e6863b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:15:59.271512 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.271517 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f6e1ffc-2fb8-4fcc-9938-097e44e6863b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2\" (UID: \"6f6e1ffc-2fb8-4fcc-9938-097e44e6863b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:15:59.271812 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.271542 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f6e1ffc-2fb8-4fcc-9938-097e44e6863b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2\" (UID: \"6f6e1ffc-2fb8-4fcc-9938-097e44e6863b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:15:59.271812 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.271583 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f6e1ffc-2fb8-4fcc-9938-097e44e6863b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2\" (UID: \"6f6e1ffc-2fb8-4fcc-9938-097e44e6863b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:15:59.271812 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.271661 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddnsj\" (UniqueName: \"kubernetes.io/projected/6f6e1ffc-2fb8-4fcc-9938-097e44e6863b-kube-api-access-ddnsj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2\" (UID: \"6f6e1ffc-2fb8-4fcc-9938-097e44e6863b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:15:59.271812 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.271703 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f6e1ffc-2fb8-4fcc-9938-097e44e6863b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2\" (UID: \"6f6e1ffc-2fb8-4fcc-9938-097e44e6863b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:15:59.372578 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.372531 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f6e1ffc-2fb8-4fcc-9938-097e44e6863b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2\" (UID: \"6f6e1ffc-2fb8-4fcc-9938-097e44e6863b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:15:59.372578 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.372580 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f6e1ffc-2fb8-4fcc-9938-097e44e6863b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2\" (UID: \"6f6e1ffc-2fb8-4fcc-9938-097e44e6863b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:15:59.372849 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.372606 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f6e1ffc-2fb8-4fcc-9938-097e44e6863b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2\" (UID: \"6f6e1ffc-2fb8-4fcc-9938-097e44e6863b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:15:59.372849 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.372716 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f6e1ffc-2fb8-4fcc-9938-097e44e6863b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2\" (UID: \"6f6e1ffc-2fb8-4fcc-9938-097e44e6863b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:15:59.372849 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.372812 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddnsj\" (UniqueName: \"kubernetes.io/projected/6f6e1ffc-2fb8-4fcc-9938-097e44e6863b-kube-api-access-ddnsj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2\" (UID: \"6f6e1ffc-2fb8-4fcc-9938-097e44e6863b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:15:59.373014 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.372859 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f6e1ffc-2fb8-4fcc-9938-097e44e6863b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2\" (UID: \"6f6e1ffc-2fb8-4fcc-9938-097e44e6863b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:15:59.373072 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.373047 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6f6e1ffc-2fb8-4fcc-9938-097e44e6863b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2\" (UID: \"6f6e1ffc-2fb8-4fcc-9938-097e44e6863b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:15:59.373131 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.373088 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6f6e1ffc-2fb8-4fcc-9938-097e44e6863b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2\" (UID: \"6f6e1ffc-2fb8-4fcc-9938-097e44e6863b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:15:59.373131 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.373114 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6f6e1ffc-2fb8-4fcc-9938-097e44e6863b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2\" (UID: \"6f6e1ffc-2fb8-4fcc-9938-097e44e6863b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:15:59.374967 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.374947 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6f6e1ffc-2fb8-4fcc-9938-097e44e6863b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2\" (UID: \"6f6e1ffc-2fb8-4fcc-9938-097e44e6863b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:15:59.375243 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.375225 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f6e1ffc-2fb8-4fcc-9938-097e44e6863b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2\" (UID: \"6f6e1ffc-2fb8-4fcc-9938-097e44e6863b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:15:59.381043 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.381022 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddnsj\" (UniqueName: \"kubernetes.io/projected/6f6e1ffc-2fb8-4fcc-9938-097e44e6863b-kube-api-access-ddnsj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2\" (UID: \"6f6e1ffc-2fb8-4fcc-9938-097e44e6863b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:15:59.490799 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.490709 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:15:59.633790 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.633758 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2"] Jul 03 10:15:59.636624 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:15:59.636598 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f6e1ffc_2fb8_4fcc_9938_097e44e6863b.slice/crio-a3a05a38ad64ce25f6eb06a7da7298c6294c92ae997b7ef0c00ea2e5dffa22e0 WatchSource:0}: Error finding container a3a05a38ad64ce25f6eb06a7da7298c6294c92ae997b7ef0c00ea2e5dffa22e0: Status 404 returned error can't find the container with id a3a05a38ad64ce25f6eb06a7da7298c6294c92ae997b7ef0c00ea2e5dffa22e0 Jul 03 10:15:59.638115 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:15:59.638098 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 10:16:00.587856 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:00.587816 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" event={"ID":"6f6e1ffc-2fb8-4fcc-9938-097e44e6863b","Type":"ContainerStarted","Data":"a3a05a38ad64ce25f6eb06a7da7298c6294c92ae997b7ef0c00ea2e5dffa22e0"} Jul 03 10:16:01.594162 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:01.594127 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" event={"ID":"6f6e1ffc-2fb8-4fcc-9938-097e44e6863b","Type":"ContainerStarted","Data":"9d37f1d4e806cbcfd059c37512634538c40a663c6ff2c819013e7c2ccf779fdc"} Jul 03 10:16:01.613359 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:01.613307 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" podStartSLOduration=1.364107204 podStartE2EDuration="2.613292115s" totalImagesPullingTime="1.249184911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:15:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:16:01.612547609 +0000 UTC m=+616.524436914" watchObservedRunningTime="2026-07-03 10:16:01.613292115 +0000 UTC m=+616.525181419" Jul 03 10:16:09.491238 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:09.491133 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:16:09.491238 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:09.491199 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:16:09.503605 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:09.503575 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:16:09.646165 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:09.646134 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2" Jul 03 10:16:17.189026 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.188995 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x"] Jul 03 10:16:17.192877 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.192856 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:17.196524 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.196491 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 03 10:16:17.214325 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.214298 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x"] Jul 03 10:16:17.231783 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.231745 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrh8x\" (UID: \"b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:17.232081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.231839 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrh8x\" (UID: \"b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:17.232081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.231867 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrh8x\" (UID: \"b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:17.232081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.231923 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrh8x\" (UID: \"b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:17.232081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.231943 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwxhm\" (UniqueName: \"kubernetes.io/projected/b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3-kube-api-access-lwxhm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrh8x\" (UID: \"b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:17.232081 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.232031 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrh8x\" (UID: \"b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:17.332692 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.332655 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrh8x\" (UID: \"b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:17.332692 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.332691 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrh8x\" (UID: \"b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:17.332940 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.332761 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrh8x\" (UID: \"b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:17.332940 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.332788 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwxhm\" (UniqueName: \"kubernetes.io/projected/b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3-kube-api-access-lwxhm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrh8x\" (UID: \"b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:17.332940 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.332878 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrh8x\" (UID: \"b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:17.332940 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.332910 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrh8x\" (UID: \"b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:17.333139 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.333123 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrh8x\" (UID: \"b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:17.333246 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.333223 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrh8x\" (UID: \"b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:17.333294 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.333246 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrh8x\" (UID: \"b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:17.335124 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.335099 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrh8x\" (UID: \"b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:17.335328 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.335312 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrh8x\" (UID: \"b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:17.345185 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.345158 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwxhm\" (UniqueName: \"kubernetes.io/projected/b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3-kube-api-access-lwxhm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mrh8x\" (UID: \"b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:17.502390 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.502292 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:17.646042 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.646005 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x"] Jul 03 10:16:17.675195 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:17.675164 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" event={"ID":"b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3","Type":"ContainerStarted","Data":"1dd99e9b88eb5d39294a7a096f147f696e27639755c690fcc776f45a05a4f631"} Jul 03 10:16:18.681609 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:18.681570 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" event={"ID":"b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3","Type":"ContainerStarted","Data":"9f5d6d30f6c3c8d764eea7bc39a105e8f01b4726f9d83e3baf1c6a963e94e178"} Jul 03 10:16:18.705288 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:18.705219 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" podStartSLOduration=1.484121843 podStartE2EDuration="1.70519523s" totalImagesPullingTime="221.073387ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:16:18.704799766 +0000 UTC m=+633.616689074" watchObservedRunningTime="2026-07-03 10:16:18.70519523 +0000 UTC m=+633.617084533" Jul 03 10:16:27.503390 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:27.503351 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:27.503910 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:27.503449 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:27.515695 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:27.515668 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:16:27.736359 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:16:27.736332 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mrh8x" Jul 03 10:18:14.831360 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:18:14.831322 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6b6d65c7c6-f479c"] Jul 03 10:18:14.831858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:18:14.831499 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-6b6d65c7c6-f479c" podUID="59a6aab5-e01f-4f27-b2e4-665842fe7bfd" containerName="authorino" containerID="cri-o://2e52d8d6c37ed9321f4c2600acdd3e1e8ca67ac85a2588f30927bc541df4837c" gracePeriod=30 Jul 03 10:18:15.086840 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:18:15.086775 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b6d65c7c6-f479c" Jul 03 10:18:15.247684 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:18:15.247648 2573 generic.go:356] "Generic (PLEG): container finished" podID="59a6aab5-e01f-4f27-b2e4-665842fe7bfd" containerID="2e52d8d6c37ed9321f4c2600acdd3e1e8ca67ac85a2588f30927bc541df4837c" exitCode=0 Jul 03 10:18:15.247874 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:18:15.247702 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b6d65c7c6-f479c" Jul 03 10:18:15.247874 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:18:15.247711 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b6d65c7c6-f479c" event={"ID":"59a6aab5-e01f-4f27-b2e4-665842fe7bfd","Type":"ContainerDied","Data":"2e52d8d6c37ed9321f4c2600acdd3e1e8ca67ac85a2588f30927bc541df4837c"} Jul 03 10:18:15.247874 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:18:15.247777 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b6d65c7c6-f479c" event={"ID":"59a6aab5-e01f-4f27-b2e4-665842fe7bfd","Type":"ContainerDied","Data":"21586822192e1cb06c16a7e43f49715b94cb2f00f4ce832d89fcda1859d97b0c"} Jul 03 10:18:15.247874 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:18:15.247798 2573 scope.go:117] "RemoveContainer" containerID="2e52d8d6c37ed9321f4c2600acdd3e1e8ca67ac85a2588f30927bc541df4837c" Jul 03 10:18:15.256050 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:18:15.256028 2573 scope.go:117] "RemoveContainer" containerID="2e52d8d6c37ed9321f4c2600acdd3e1e8ca67ac85a2588f30927bc541df4837c" Jul 03 10:18:15.256290 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:18:15.256273 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e52d8d6c37ed9321f4c2600acdd3e1e8ca67ac85a2588f30927bc541df4837c\": container with ID starting with 2e52d8d6c37ed9321f4c2600acdd3e1e8ca67ac85a2588f30927bc541df4837c not found: ID does not exist" containerID="2e52d8d6c37ed9321f4c2600acdd3e1e8ca67ac85a2588f30927bc541df4837c" Jul 03 10:18:15.256354 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:18:15.256297 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e52d8d6c37ed9321f4c2600acdd3e1e8ca67ac85a2588f30927bc541df4837c"} err="failed to get container status \"2e52d8d6c37ed9321f4c2600acdd3e1e8ca67ac85a2588f30927bc541df4837c\": rpc error: code = NotFound desc = could not find container \"2e52d8d6c37ed9321f4c2600acdd3e1e8ca67ac85a2588f30927bc541df4837c\": container with ID starting with 2e52d8d6c37ed9321f4c2600acdd3e1e8ca67ac85a2588f30927bc541df4837c not found: ID does not exist" Jul 03 10:18:15.262638 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:18:15.262616 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59s89\" (UniqueName: \"kubernetes.io/projected/59a6aab5-e01f-4f27-b2e4-665842fe7bfd-kube-api-access-59s89\") pod \"59a6aab5-e01f-4f27-b2e4-665842fe7bfd\" (UID: \"59a6aab5-e01f-4f27-b2e4-665842fe7bfd\") " Jul 03 10:18:15.262753 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:18:15.262676 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/59a6aab5-e01f-4f27-b2e4-665842fe7bfd-tls-cert\") pod \"59a6aab5-e01f-4f27-b2e4-665842fe7bfd\" (UID: \"59a6aab5-e01f-4f27-b2e4-665842fe7bfd\") " Jul 03 10:18:15.264632 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:18:15.264606 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59a6aab5-e01f-4f27-b2e4-665842fe7bfd-kube-api-access-59s89" (OuterVolumeSpecName: "kube-api-access-59s89") pod "59a6aab5-e01f-4f27-b2e4-665842fe7bfd" (UID: "59a6aab5-e01f-4f27-b2e4-665842fe7bfd"). InnerVolumeSpecName "kube-api-access-59s89". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:18:15.273426 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:18:15.273398 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59a6aab5-e01f-4f27-b2e4-665842fe7bfd-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "59a6aab5-e01f-4f27-b2e4-665842fe7bfd" (UID: "59a6aab5-e01f-4f27-b2e4-665842fe7bfd"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:18:15.364118 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:18:15.364034 2573 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/59a6aab5-e01f-4f27-b2e4-665842fe7bfd-tls-cert\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:18:15.364118 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:18:15.364065 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59s89\" (UniqueName: \"kubernetes.io/projected/59a6aab5-e01f-4f27-b2e4-665842fe7bfd-kube-api-access-59s89\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:18:15.586197 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:18:15.586167 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6b6d65c7c6-f479c"] Jul 03 10:18:15.653168 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:18:15.653140 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6b6d65c7c6-f479c"] Jul 03 10:18:16.563316 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:18:16.563281 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59a6aab5-e01f-4f27-b2e4-665842fe7bfd" path="/var/lib/kubelet/pods/59a6aab5-e01f-4f27-b2e4-665842fe7bfd/volumes" Jul 03 10:30:00.199825 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:00.199681 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29717910-4thq5"] Jul 03 10:30:00.200392 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:00.200182 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59a6aab5-e01f-4f27-b2e4-665842fe7bfd" containerName="authorino" Jul 03 10:30:00.200392 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:00.200196 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="59a6aab5-e01f-4f27-b2e4-665842fe7bfd" containerName="authorino" Jul 03 10:30:00.200392 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:00.200273 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="59a6aab5-e01f-4f27-b2e4-665842fe7bfd" containerName="authorino" Jul 03 10:30:00.203315 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:00.203297 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717910-4thq5" Jul 03 10:30:00.210399 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:00.210380 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-kc4sf\"" Jul 03 10:30:00.258112 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:00.258075 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717910-4thq5"] Jul 03 10:30:00.322252 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:00.322212 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5ff5\" (UniqueName: \"kubernetes.io/projected/f55e7c56-27c2-4783-a582-5689689477fd-kube-api-access-s5ff5\") pod \"maas-api-key-cleanup-29717910-4thq5\" (UID: \"f55e7c56-27c2-4783-a582-5689689477fd\") " pod="opendatahub/maas-api-key-cleanup-29717910-4thq5" Jul 03 10:30:00.422868 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:00.422825 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5ff5\" (UniqueName: \"kubernetes.io/projected/f55e7c56-27c2-4783-a582-5689689477fd-kube-api-access-s5ff5\") pod \"maas-api-key-cleanup-29717910-4thq5\" (UID: \"f55e7c56-27c2-4783-a582-5689689477fd\") " pod="opendatahub/maas-api-key-cleanup-29717910-4thq5" Jul 03 10:30:00.439307 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:00.439272 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5ff5\" (UniqueName: \"kubernetes.io/projected/f55e7c56-27c2-4783-a582-5689689477fd-kube-api-access-s5ff5\") pod \"maas-api-key-cleanup-29717910-4thq5\" (UID: \"f55e7c56-27c2-4783-a582-5689689477fd\") " pod="opendatahub/maas-api-key-cleanup-29717910-4thq5" Jul 03 10:30:00.513553 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:00.513475 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717910-4thq5" Jul 03 10:30:00.657271 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:00.657248 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717910-4thq5"] Jul 03 10:30:00.659996 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:30:00.659967 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf55e7c56_27c2_4783_a582_5689689477fd.slice/crio-450424f3a49f374a6304c89046b045227b00c81f96ebc7ac690cd2f8c68af8a2 WatchSource:0}: Error finding container 450424f3a49f374a6304c89046b045227b00c81f96ebc7ac690cd2f8c68af8a2: Status 404 returned error can't find the container with id 450424f3a49f374a6304c89046b045227b00c81f96ebc7ac690cd2f8c68af8a2 Jul 03 10:30:00.662020 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:00.661999 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 10:30:01.657361 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:01.657327 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717910-4thq5" event={"ID":"f55e7c56-27c2-4783-a582-5689689477fd","Type":"ContainerStarted","Data":"450424f3a49f374a6304c89046b045227b00c81f96ebc7ac690cd2f8c68af8a2"} Jul 03 10:30:15.727655 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:15.727617 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717910-4thq5" event={"ID":"f55e7c56-27c2-4783-a582-5689689477fd","Type":"ContainerStarted","Data":"afcdcfed83e669abbdf9e72a943d501f986bb0c8bb1e2b460c542387b26bb518"} Jul 03 10:30:15.752041 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:15.751978 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29717910-4thq5" podStartSLOduration=1.1542486809999999 podStartE2EDuration="15.751958118s" totalImagesPullingTime="14.597709437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:30:15.751033327 +0000 UTC m=+1470.662922625" watchObservedRunningTime="2026-07-03 10:30:15.751958118 +0000 UTC m=+1470.663847422" Jul 03 10:30:16.734170 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:16.734136 2573 generic.go:356] "Generic (PLEG): container finished" podID="f55e7c56-27c2-4783-a582-5689689477fd" containerID="afcdcfed83e669abbdf9e72a943d501f986bb0c8bb1e2b460c542387b26bb518" exitCode=0 Jul 03 10:30:16.734561 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:16.734183 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717910-4thq5" event={"ID":"f55e7c56-27c2-4783-a582-5689689477fd","Type":"ContainerDied","Data":"afcdcfed83e669abbdf9e72a943d501f986bb0c8bb1e2b460c542387b26bb518"} Jul 03 10:30:17.866375 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:17.866350 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717910-4thq5" Jul 03 10:30:17.988962 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:17.988929 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5ff5\" (UniqueName: \"kubernetes.io/projected/f55e7c56-27c2-4783-a582-5689689477fd-kube-api-access-s5ff5\") pod \"f55e7c56-27c2-4783-a582-5689689477fd\" (UID: \"f55e7c56-27c2-4783-a582-5689689477fd\") " Jul 03 10:30:17.991073 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:17.991006 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f55e7c56-27c2-4783-a582-5689689477fd-kube-api-access-s5ff5" (OuterVolumeSpecName: "kube-api-access-s5ff5") pod "f55e7c56-27c2-4783-a582-5689689477fd" (UID: "f55e7c56-27c2-4783-a582-5689689477fd"). InnerVolumeSpecName "kube-api-access-s5ff5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:30:18.090611 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:18.090575 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5ff5\" (UniqueName: \"kubernetes.io/projected/f55e7c56-27c2-4783-a582-5689689477fd-kube-api-access-s5ff5\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:30:18.744913 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:18.744882 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717910-4thq5" Jul 03 10:30:18.745082 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:18.744919 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717910-4thq5" event={"ID":"f55e7c56-27c2-4783-a582-5689689477fd","Type":"ContainerDied","Data":"450424f3a49f374a6304c89046b045227b00c81f96ebc7ac690cd2f8c68af8a2"} Jul 03 10:30:18.745082 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:30:18.744963 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="450424f3a49f374a6304c89046b045227b00c81f96ebc7ac690cd2f8c68af8a2" Jul 03 10:34:33.701238 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.701160 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr"] Jul 03 10:34:33.701671 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.701553 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f55e7c56-27c2-4783-a582-5689689477fd" containerName="cleanup" Jul 03 10:34:33.701671 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.701567 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f55e7c56-27c2-4783-a582-5689689477fd" containerName="cleanup" Jul 03 10:34:33.701671 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.701638 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f55e7c56-27c2-4783-a582-5689689477fd" containerName="cleanup" Jul 03 10:34:33.704677 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.704662 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:33.712108 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.712088 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-dcde78d2\"/\"e2e-lifecycle-model-dcde78d2-kserve-self-signed-certs\"" Jul 03 10:34:33.712218 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.712104 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-dcde78d2\"/\"default-dockercfg-9dt4m\"" Jul 03 10:34:33.712218 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.712135 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-dcde78d2\"/\"openshift-service-ca.crt\"" Jul 03 10:34:33.712218 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.712142 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-dcde78d2\"/\"kube-root-ca.crt\"" Jul 03 10:34:33.717724 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.717704 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr"] Jul 03 10:34:33.813683 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.813649 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-dshm\") pod \"e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:33.813683 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.813685 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-model-cache\") pod \"e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:33.813924 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.813716 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgbtk\" (UniqueName: \"kubernetes.io/projected/88a92cb2-be51-4404-affb-876d1178fd0a-kube-api-access-tgbtk\") pod \"e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:33.813924 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.813830 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88a92cb2-be51-4404-affb-876d1178fd0a-tls-certs\") pod \"e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:33.813997 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.813934 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-tmp-dir\") pod \"e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:33.813997 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.813970 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-home\") pod \"e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:33.914894 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.914859 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-home\") pod \"e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:33.914894 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.914910 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-dshm\") pod \"e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:33.915154 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.914929 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-model-cache\") pod \"e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:33.915154 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.914959 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgbtk\" (UniqueName: \"kubernetes.io/projected/88a92cb2-be51-4404-affb-876d1178fd0a-kube-api-access-tgbtk\") pod \"e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:33.915154 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.914983 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88a92cb2-be51-4404-affb-876d1178fd0a-tls-certs\") pod \"e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:33.915154 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.915039 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-tmp-dir\") pod \"e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:33.915411 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.915389 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-model-cache\") pod \"e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:33.915465 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.915422 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-home\") pod \"e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:33.915502 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.915459 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-tmp-dir\") pod \"e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:33.917150 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.917127 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-dshm\") pod \"e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:33.917519 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.917498 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88a92cb2-be51-4404-affb-876d1178fd0a-tls-certs\") pod \"e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:33.924273 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:33.924251 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgbtk\" (UniqueName: \"kubernetes.io/projected/88a92cb2-be51-4404-affb-876d1178fd0a-kube-api-access-tgbtk\") pod \"e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:34.014272 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:34.014186 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:34.167304 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:34.167270 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr"] Jul 03 10:34:34.172616 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:34:34.172576 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88a92cb2_be51_4404_affb_876d1178fd0a.slice/crio-3951b38e3cf99c12df9350540ec87ce5d7ee81e1a9aea6f2cec2a6070cdc592d WatchSource:0}: Error finding container 3951b38e3cf99c12df9350540ec87ce5d7ee81e1a9aea6f2cec2a6070cdc592d: Status 404 returned error can't find the container with id 3951b38e3cf99c12df9350540ec87ce5d7ee81e1a9aea6f2cec2a6070cdc592d Jul 03 10:34:34.995955 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:34.995921 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" event={"ID":"88a92cb2-be51-4404-affb-876d1178fd0a","Type":"ContainerStarted","Data":"b6360fe6825aced6021162dad2c971743b91b4c436aa8fe67aad266b0ea42f99"} Jul 03 10:34:34.995955 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:34.995961 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" event={"ID":"88a92cb2-be51-4404-affb-876d1178fd0a","Type":"ContainerStarted","Data":"3951b38e3cf99c12df9350540ec87ce5d7ee81e1a9aea6f2cec2a6070cdc592d"} Jul 03 10:34:35.019246 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:35.019188 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" podStartSLOduration=2.019171577 podStartE2EDuration="2.019171577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:34:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:34:35.017789212 +0000 UTC m=+1729.929678516" watchObservedRunningTime="2026-07-03 10:34:35.019171577 +0000 UTC m=+1729.931060879" Jul 03 10:34:44.014838 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:44.014795 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:44.015301 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:44.014888 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:44.027773 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:44.027745 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:34:44.049593 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:34:44.049563 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:35:07.128807 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:07.128772 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr"] Jul 03 10:35:22.149168 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:22.149126 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" podUID="88a92cb2-be51-4404-affb-876d1178fd0a" containerName="main" containerID="cri-o://b6360fe6825aced6021162dad2c971743b91b4c436aa8fe67aad266b0ea42f99" gracePeriod=45 Jul 03 10:35:22.399784 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:22.399700 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:35:22.477845 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:22.477814 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88a92cb2-be51-4404-affb-876d1178fd0a-tls-certs\") pod \"88a92cb2-be51-4404-affb-876d1178fd0a\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " Jul 03 10:35:22.478029 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:22.477871 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-home\") pod \"88a92cb2-be51-4404-affb-876d1178fd0a\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " Jul 03 10:35:22.478029 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:22.477912 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-model-cache\") pod \"88a92cb2-be51-4404-affb-876d1178fd0a\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " Jul 03 10:35:22.478029 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:22.477951 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-dshm\") pod \"88a92cb2-be51-4404-affb-876d1178fd0a\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " Jul 03 10:35:22.478029 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:22.478006 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-tmp-dir\") pod \"88a92cb2-be51-4404-affb-876d1178fd0a\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " Jul 03 10:35:22.478233 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:22.478037 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgbtk\" (UniqueName: \"kubernetes.io/projected/88a92cb2-be51-4404-affb-876d1178fd0a-kube-api-access-tgbtk\") pod \"88a92cb2-be51-4404-affb-876d1178fd0a\" (UID: \"88a92cb2-be51-4404-affb-876d1178fd0a\") " Jul 03 10:35:22.478233 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:22.478198 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-model-cache" (OuterVolumeSpecName: "model-cache") pod "88a92cb2-be51-4404-affb-876d1178fd0a" (UID: "88a92cb2-be51-4404-affb-876d1178fd0a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:35:22.478233 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:22.478208 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-home" (OuterVolumeSpecName: "home") pod "88a92cb2-be51-4404-affb-876d1178fd0a" (UID: "88a92cb2-be51-4404-affb-876d1178fd0a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:35:22.478361 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:22.478247 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "88a92cb2-be51-4404-affb-876d1178fd0a" (UID: "88a92cb2-be51-4404-affb-876d1178fd0a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:35:22.478402 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:22.478368 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-home\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:35:22.478402 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:22.478382 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-model-cache\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:35:22.478402 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:22.478392 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-tmp-dir\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:35:22.480087 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:22.480062 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-dshm" (OuterVolumeSpecName: "dshm") pod "88a92cb2-be51-4404-affb-876d1178fd0a" (UID: "88a92cb2-be51-4404-affb-876d1178fd0a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:35:22.480193 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:22.480113 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88a92cb2-be51-4404-affb-876d1178fd0a-kube-api-access-tgbtk" (OuterVolumeSpecName: "kube-api-access-tgbtk") pod "88a92cb2-be51-4404-affb-876d1178fd0a" (UID: "88a92cb2-be51-4404-affb-876d1178fd0a"). InnerVolumeSpecName "kube-api-access-tgbtk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:35:22.480193 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:22.480144 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88a92cb2-be51-4404-affb-876d1178fd0a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "88a92cb2-be51-4404-affb-876d1178fd0a" (UID: "88a92cb2-be51-4404-affb-876d1178fd0a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:35:22.579451 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:22.579423 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgbtk\" (UniqueName: \"kubernetes.io/projected/88a92cb2-be51-4404-affb-876d1178fd0a-kube-api-access-tgbtk\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:35:22.579451 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:22.579451 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88a92cb2-be51-4404-affb-876d1178fd0a-tls-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:35:22.579651 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:22.579464 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88a92cb2-be51-4404-affb-876d1178fd0a-dshm\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:35:23.240789 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:23.240755 2573 generic.go:356] "Generic (PLEG): container finished" podID="88a92cb2-be51-4404-affb-876d1178fd0a" containerID="b6360fe6825aced6021162dad2c971743b91b4c436aa8fe67aad266b0ea42f99" exitCode=0 Jul 03 10:35:23.241289 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:23.240804 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" Jul 03 10:35:23.241289 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:23.240835 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" event={"ID":"88a92cb2-be51-4404-affb-876d1178fd0a","Type":"ContainerDied","Data":"b6360fe6825aced6021162dad2c971743b91b4c436aa8fe67aad266b0ea42f99"} Jul 03 10:35:23.241289 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:23.240881 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr" event={"ID":"88a92cb2-be51-4404-affb-876d1178fd0a","Type":"ContainerDied","Data":"3951b38e3cf99c12df9350540ec87ce5d7ee81e1a9aea6f2cec2a6070cdc592d"} Jul 03 10:35:23.241289 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:23.240903 2573 scope.go:117] "RemoveContainer" containerID="b6360fe6825aced6021162dad2c971743b91b4c436aa8fe67aad266b0ea42f99" Jul 03 10:35:23.249426 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:23.249407 2573 scope.go:117] "RemoveContainer" containerID="b6360fe6825aced6021162dad2c971743b91b4c436aa8fe67aad266b0ea42f99" Jul 03 10:35:23.249709 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:35:23.249691 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6360fe6825aced6021162dad2c971743b91b4c436aa8fe67aad266b0ea42f99\": container with ID starting with b6360fe6825aced6021162dad2c971743b91b4c436aa8fe67aad266b0ea42f99 not found: ID does not exist" containerID="b6360fe6825aced6021162dad2c971743b91b4c436aa8fe67aad266b0ea42f99" Jul 03 10:35:23.249771 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:23.249718 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6360fe6825aced6021162dad2c971743b91b4c436aa8fe67aad266b0ea42f99"} err="failed to get container status \"b6360fe6825aced6021162dad2c971743b91b4c436aa8fe67aad266b0ea42f99\": rpc error: code = NotFound desc = could not find container \"b6360fe6825aced6021162dad2c971743b91b4c436aa8fe67aad266b0ea42f99\": container with ID starting with b6360fe6825aced6021162dad2c971743b91b4c436aa8fe67aad266b0ea42f99 not found: ID does not exist" Jul 03 10:35:23.264900 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:23.264875 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr"] Jul 03 10:35:23.272048 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:23.272025 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-dcde78d2/e2e-lifecycle-model-dcde78d2-kserve-6774b97dc-flsmr"] Jul 03 10:35:24.563840 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:35:24.563800 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88a92cb2-be51-4404-affb-876d1178fd0a" path="/var/lib/kubelet/pods/88a92cb2-be51-4404-affb-876d1178fd0a/volumes" Jul 03 10:37:03.453391 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:03.453358 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b"] Jul 03 10:37:03.453825 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:03.453799 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88a92cb2-be51-4404-affb-876d1178fd0a" containerName="main" Jul 03 10:37:03.453825 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:03.453811 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="88a92cb2-be51-4404-affb-876d1178fd0a" containerName="main" Jul 03 10:37:03.453900 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:03.453892 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="88a92cb2-be51-4404-affb-876d1178fd0a" containerName="main" Jul 03 10:37:03.456972 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:03.456955 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" Jul 03 10:37:03.461480 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:03.461456 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-7436d1-serving-cert\"" Jul 03 10:37:03.461588 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:03.461521 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-kc4sf\"" Jul 03 10:37:03.491036 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:03.491009 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b"] Jul 03 10:37:03.537381 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:03.537343 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f6f4130-52fa-46dc-a127-641a23eddf59-maas-api-tls\") pod \"maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b\" (UID: \"4f6f4130-52fa-46dc-a127-641a23eddf59\") " pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" Jul 03 10:37:03.537587 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:03.537488 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2hp7\" (UniqueName: \"kubernetes.io/projected/4f6f4130-52fa-46dc-a127-641a23eddf59-kube-api-access-k2hp7\") pod \"maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b\" (UID: \"4f6f4130-52fa-46dc-a127-641a23eddf59\") " pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" Jul 03 10:37:03.537587 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:03.537528 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f6f4130-52fa-46dc-a127-641a23eddf59-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b\" (UID: \"4f6f4130-52fa-46dc-a127-641a23eddf59\") " pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" Jul 03 10:37:03.638538 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:03.638494 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2hp7\" (UniqueName: \"kubernetes.io/projected/4f6f4130-52fa-46dc-a127-641a23eddf59-kube-api-access-k2hp7\") pod \"maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b\" (UID: \"4f6f4130-52fa-46dc-a127-641a23eddf59\") " pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" Jul 03 10:37:03.638538 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:03.638540 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f6f4130-52fa-46dc-a127-641a23eddf59-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b\" (UID: \"4f6f4130-52fa-46dc-a127-641a23eddf59\") " pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" Jul 03 10:37:03.638839 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:03.638582 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f6f4130-52fa-46dc-a127-641a23eddf59-maas-api-tls\") pod \"maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b\" (UID: \"4f6f4130-52fa-46dc-a127-641a23eddf59\") " pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" Jul 03 10:37:03.639301 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:03.639275 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f6f4130-52fa-46dc-a127-641a23eddf59-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b\" (UID: \"4f6f4130-52fa-46dc-a127-641a23eddf59\") " pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" Jul 03 10:37:03.641161 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:03.641141 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f6f4130-52fa-46dc-a127-641a23eddf59-maas-api-tls\") pod \"maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b\" (UID: \"4f6f4130-52fa-46dc-a127-641a23eddf59\") " pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" Jul 03 10:37:03.652791 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:03.652767 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2hp7\" (UniqueName: \"kubernetes.io/projected/4f6f4130-52fa-46dc-a127-641a23eddf59-kube-api-access-k2hp7\") pod \"maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b\" (UID: \"4f6f4130-52fa-46dc-a127-641a23eddf59\") " pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" Jul 03 10:37:03.767361 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:03.767264 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" Jul 03 10:37:03.927186 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:03.927163 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b"] Jul 03 10:37:03.929109 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:37:03.929078 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f6f4130_52fa_46dc_a127_641a23eddf59.slice/crio-44f2fa21eb07d5f32aa61e71b918bb8b4151ce001ea977153c814760303bc608 WatchSource:0}: Error finding container 44f2fa21eb07d5f32aa61e71b918bb8b4151ce001ea977153c814760303bc608: Status 404 returned error can't find the container with id 44f2fa21eb07d5f32aa61e71b918bb8b4151ce001ea977153c814760303bc608 Jul 03 10:37:03.930433 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:03.930412 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 10:37:04.745082 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:04.744995 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" event={"ID":"4f6f4130-52fa-46dc-a127-641a23eddf59","Type":"ContainerStarted","Data":"44f2fa21eb07d5f32aa61e71b918bb8b4151ce001ea977153c814760303bc608"} Jul 03 10:37:05.757842 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:05.757695 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" event={"ID":"4f6f4130-52fa-46dc-a127-641a23eddf59","Type":"ContainerStarted","Data":"a1803a98dd97b1c10d76ba0f2545362ba08f60c323183c25554763e0e15d5e66"} Jul 03 10:37:05.757842 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:05.757822 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" Jul 03 10:37:05.789952 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:05.789894 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" podStartSLOduration=1.049828597 podStartE2EDuration="2.789877702s" totalImagesPullingTime="1.740049105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:37:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:37:05.788179815 +0000 UTC m=+1880.700069119" watchObservedRunningTime="2026-07-03 10:37:05.789877702 +0000 UTC m=+1880.701767006" Jul 03 10:37:11.769149 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:11.769076 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" Jul 03 10:37:57.820429 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:57.820389 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg"] Jul 03 10:37:57.823989 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:57.823971 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:37:57.830162 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:57.830132 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7436d1\"/\"kube-root-ca.crt\"" Jul 03 10:37:57.831523 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:57.831500 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7436d1\"/\"openshift-service-ca.crt\"" Jul 03 10:37:57.831630 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:57.831587 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7436d1\"/\"test-model-7436d1-kserve-self-signed-certs\"" Jul 03 10:37:57.831807 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:57.831636 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7436d1\"/\"default-dockercfg-pv2m8\"" Jul 03 10:37:57.841303 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:57.841282 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg"] Jul 03 10:37:57.956441 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:57.956401 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36579fb9-7857-493a-9d88-e7b178824735-tls-certs\") pod \"test-model-7436d1-kserve-7cf9554bcf-tl6jg\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:37:57.956441 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:57.956446 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdqgj\" (UniqueName: \"kubernetes.io/projected/36579fb9-7857-493a-9d88-e7b178824735-kube-api-access-xdqgj\") pod \"test-model-7436d1-kserve-7cf9554bcf-tl6jg\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:37:57.956680 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:57.956473 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-tmp-dir\") pod \"test-model-7436d1-kserve-7cf9554bcf-tl6jg\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:37:57.956680 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:57.956549 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-model-cache\") pod \"test-model-7436d1-kserve-7cf9554bcf-tl6jg\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:37:57.956680 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:57.956620 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-dshm\") pod \"test-model-7436d1-kserve-7cf9554bcf-tl6jg\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:37:57.956680 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:57.956651 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-home\") pod \"test-model-7436d1-kserve-7cf9554bcf-tl6jg\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:37:58.057978 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:58.057944 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36579fb9-7857-493a-9d88-e7b178824735-tls-certs\") pod \"test-model-7436d1-kserve-7cf9554bcf-tl6jg\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:37:58.058166 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:58.057986 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdqgj\" (UniqueName: \"kubernetes.io/projected/36579fb9-7857-493a-9d88-e7b178824735-kube-api-access-xdqgj\") pod \"test-model-7436d1-kserve-7cf9554bcf-tl6jg\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:37:58.058166 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:58.058021 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-tmp-dir\") pod \"test-model-7436d1-kserve-7cf9554bcf-tl6jg\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:37:58.058166 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:58.058047 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-model-cache\") pod \"test-model-7436d1-kserve-7cf9554bcf-tl6jg\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:37:58.058166 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:58.058098 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-dshm\") pod \"test-model-7436d1-kserve-7cf9554bcf-tl6jg\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:37:58.058166 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:58.058144 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-home\") pod \"test-model-7436d1-kserve-7cf9554bcf-tl6jg\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:37:58.058464 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:58.058439 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-tmp-dir\") pod \"test-model-7436d1-kserve-7cf9554bcf-tl6jg\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:37:58.058515 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:58.058462 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-model-cache\") pod \"test-model-7436d1-kserve-7cf9554bcf-tl6jg\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:37:58.058564 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:58.058543 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-home\") pod \"test-model-7436d1-kserve-7cf9554bcf-tl6jg\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:37:58.060397 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:58.060362 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-dshm\") pod \"test-model-7436d1-kserve-7cf9554bcf-tl6jg\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:37:58.060605 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:58.060575 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36579fb9-7857-493a-9d88-e7b178824735-tls-certs\") pod \"test-model-7436d1-kserve-7cf9554bcf-tl6jg\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:37:58.068544 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:58.068518 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdqgj\" (UniqueName: \"kubernetes.io/projected/36579fb9-7857-493a-9d88-e7b178824735-kube-api-access-xdqgj\") pod \"test-model-7436d1-kserve-7cf9554bcf-tl6jg\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:37:58.133525 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:58.133493 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:37:58.277308 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:37:58.277271 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36579fb9_7857_493a_9d88_e7b178824735.slice/crio-448c693ce7d9421a93433710b5ed6c237e3fcb29a6a7de151f1d736f700d0a43 WatchSource:0}: Error finding container 448c693ce7d9421a93433710b5ed6c237e3fcb29a6a7de151f1d736f700d0a43: Status 404 returned error can't find the container with id 448c693ce7d9421a93433710b5ed6c237e3fcb29a6a7de151f1d736f700d0a43 Jul 03 10:37:58.278479 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:58.278453 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg"] Jul 03 10:37:59.020924 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:59.020884 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" event={"ID":"36579fb9-7857-493a-9d88-e7b178824735","Type":"ContainerStarted","Data":"5d50a49145fc5afb4f2d0409d799d44df8a0c49ff20c450630cd2320b0d0ca4b"} Jul 03 10:37:59.021343 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:59.020936 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" event={"ID":"36579fb9-7857-493a-9d88-e7b178824735","Type":"ContainerStarted","Data":"448c693ce7d9421a93433710b5ed6c237e3fcb29a6a7de151f1d736f700d0a43"} Jul 03 10:37:59.041458 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:37:59.041412 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" podStartSLOduration=2.04139723 podStartE2EDuration="2.04139723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:37:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:37:59.040053334 +0000 UTC m=+1933.951942640" watchObservedRunningTime="2026-07-03 10:37:59.04139723 +0000 UTC m=+1933.953286531" Jul 03 10:38:08.134596 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:08.134555 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:38:08.134596 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:08.134600 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:38:08.147151 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:08.147124 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:38:09.081152 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:09.081122 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:38:17.685905 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:17.685869 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg"] Jul 03 10:38:32.705712 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:32.705654 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" podUID="36579fb9-7857-493a-9d88-e7b178824735" containerName="main" containerID="cri-o://5d50a49145fc5afb4f2d0409d799d44df8a0c49ff20c450630cd2320b0d0ca4b" gracePeriod=45 Jul 03 10:38:32.960205 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:32.960148 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:38:32.994027 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:32.993999 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-dshm\") pod \"36579fb9-7857-493a-9d88-e7b178824735\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " Jul 03 10:38:32.994199 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:32.994060 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-tmp-dir\") pod \"36579fb9-7857-493a-9d88-e7b178824735\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " Jul 03 10:38:32.994199 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:32.994103 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-model-cache\") pod \"36579fb9-7857-493a-9d88-e7b178824735\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " Jul 03 10:38:32.994199 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:32.994120 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-home\") pod \"36579fb9-7857-493a-9d88-e7b178824735\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " Jul 03 10:38:32.994199 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:32.994142 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdqgj\" (UniqueName: \"kubernetes.io/projected/36579fb9-7857-493a-9d88-e7b178824735-kube-api-access-xdqgj\") pod \"36579fb9-7857-493a-9d88-e7b178824735\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " Jul 03 10:38:32.994199 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:32.994169 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36579fb9-7857-493a-9d88-e7b178824735-tls-certs\") pod \"36579fb9-7857-493a-9d88-e7b178824735\" (UID: \"36579fb9-7857-493a-9d88-e7b178824735\") " Jul 03 10:38:32.994482 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:32.994381 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "36579fb9-7857-493a-9d88-e7b178824735" (UID: "36579fb9-7857-493a-9d88-e7b178824735"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:38:32.994482 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:32.994393 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-model-cache" (OuterVolumeSpecName: "model-cache") pod "36579fb9-7857-493a-9d88-e7b178824735" (UID: "36579fb9-7857-493a-9d88-e7b178824735"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:38:32.994482 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:32.994456 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-home" (OuterVolumeSpecName: "home") pod "36579fb9-7857-493a-9d88-e7b178824735" (UID: "36579fb9-7857-493a-9d88-e7b178824735"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:38:32.994895 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:32.994582 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-tmp-dir\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:38:32.994895 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:32.994603 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-model-cache\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:38:32.994895 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:32.994616 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-home\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:38:32.996487 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:32.996459 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-dshm" (OuterVolumeSpecName: "dshm") pod "36579fb9-7857-493a-9d88-e7b178824735" (UID: "36579fb9-7857-493a-9d88-e7b178824735"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:38:32.996487 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:32.996478 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36579fb9-7857-493a-9d88-e7b178824735-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "36579fb9-7857-493a-9d88-e7b178824735" (UID: "36579fb9-7857-493a-9d88-e7b178824735"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:38:32.996668 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:32.996510 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36579fb9-7857-493a-9d88-e7b178824735-kube-api-access-xdqgj" (OuterVolumeSpecName: "kube-api-access-xdqgj") pod "36579fb9-7857-493a-9d88-e7b178824735" (UID: "36579fb9-7857-493a-9d88-e7b178824735"). InnerVolumeSpecName "kube-api-access-xdqgj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:38:33.095545 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:33.095505 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36579fb9-7857-493a-9d88-e7b178824735-dshm\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:38:33.095545 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:33.095537 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdqgj\" (UniqueName: \"kubernetes.io/projected/36579fb9-7857-493a-9d88-e7b178824735-kube-api-access-xdqgj\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:38:33.095545 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:33.095548 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36579fb9-7857-493a-9d88-e7b178824735-tls-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:38:33.186313 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:33.186280 2573 generic.go:356] "Generic (PLEG): container finished" podID="36579fb9-7857-493a-9d88-e7b178824735" containerID="5d50a49145fc5afb4f2d0409d799d44df8a0c49ff20c450630cd2320b0d0ca4b" exitCode=0 Jul 03 10:38:33.186500 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:33.186330 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" Jul 03 10:38:33.186500 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:33.186364 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" event={"ID":"36579fb9-7857-493a-9d88-e7b178824735","Type":"ContainerDied","Data":"5d50a49145fc5afb4f2d0409d799d44df8a0c49ff20c450630cd2320b0d0ca4b"} Jul 03 10:38:33.186500 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:33.186411 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg" event={"ID":"36579fb9-7857-493a-9d88-e7b178824735","Type":"ContainerDied","Data":"448c693ce7d9421a93433710b5ed6c237e3fcb29a6a7de151f1d736f700d0a43"} Jul 03 10:38:33.186500 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:33.186430 2573 scope.go:117] "RemoveContainer" containerID="5d50a49145fc5afb4f2d0409d799d44df8a0c49ff20c450630cd2320b0d0ca4b" Jul 03 10:38:33.195517 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:33.195497 2573 scope.go:117] "RemoveContainer" containerID="5d50a49145fc5afb4f2d0409d799d44df8a0c49ff20c450630cd2320b0d0ca4b" Jul 03 10:38:33.195802 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:38:33.195777 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d50a49145fc5afb4f2d0409d799d44df8a0c49ff20c450630cd2320b0d0ca4b\": container with ID starting with 5d50a49145fc5afb4f2d0409d799d44df8a0c49ff20c450630cd2320b0d0ca4b not found: ID does not exist" containerID="5d50a49145fc5afb4f2d0409d799d44df8a0c49ff20c450630cd2320b0d0ca4b" Jul 03 10:38:33.195887 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:33.195812 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d50a49145fc5afb4f2d0409d799d44df8a0c49ff20c450630cd2320b0d0ca4b"} err="failed to get container status \"5d50a49145fc5afb4f2d0409d799d44df8a0c49ff20c450630cd2320b0d0ca4b\": rpc error: code = NotFound desc = could not find container \"5d50a49145fc5afb4f2d0409d799d44df8a0c49ff20c450630cd2320b0d0ca4b\": container with ID starting with 5d50a49145fc5afb4f2d0409d799d44df8a0c49ff20c450630cd2320b0d0ca4b not found: ID does not exist" Jul 03 10:38:33.224920 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:33.224849 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg"] Jul 03 10:38:33.230844 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:33.230816 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-7436d1/test-model-7436d1-kserve-7cf9554bcf-tl6jg"] Jul 03 10:38:34.565167 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:38:34.565133 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36579fb9-7857-493a-9d88-e7b178824735" path="/var/lib/kubelet/pods/36579fb9-7857-493a-9d88-e7b178824735/volumes" Jul 03 10:39:03.240782 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:03.240748 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b"] Jul 03 10:39:03.241217 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:03.241000 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" podUID="4f6f4130-52fa-46dc-a127-641a23eddf59" containerName="maas-api" containerID="cri-o://a1803a98dd97b1c10d76ba0f2545362ba08f60c323183c25554763e0e15d5e66" gracePeriod=30 Jul 03 10:39:03.490911 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:03.490850 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" Jul 03 10:39:03.587983 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:03.587948 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f6f4130-52fa-46dc-a127-641a23eddf59-openshift-service-ca\") pod \"4f6f4130-52fa-46dc-a127-641a23eddf59\" (UID: \"4f6f4130-52fa-46dc-a127-641a23eddf59\") " Jul 03 10:39:03.588157 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:03.588009 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f6f4130-52fa-46dc-a127-641a23eddf59-maas-api-tls\") pod \"4f6f4130-52fa-46dc-a127-641a23eddf59\" (UID: \"4f6f4130-52fa-46dc-a127-641a23eddf59\") " Jul 03 10:39:03.588157 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:03.588033 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2hp7\" (UniqueName: \"kubernetes.io/projected/4f6f4130-52fa-46dc-a127-641a23eddf59-kube-api-access-k2hp7\") pod \"4f6f4130-52fa-46dc-a127-641a23eddf59\" (UID: \"4f6f4130-52fa-46dc-a127-641a23eddf59\") " Jul 03 10:39:03.588386 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:03.588356 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f6f4130-52fa-46dc-a127-641a23eddf59-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4f6f4130-52fa-46dc-a127-641a23eddf59" (UID: "4f6f4130-52fa-46dc-a127-641a23eddf59"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:39:03.590144 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:03.590115 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f6f4130-52fa-46dc-a127-641a23eddf59-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4f6f4130-52fa-46dc-a127-641a23eddf59" (UID: "4f6f4130-52fa-46dc-a127-641a23eddf59"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:39:03.590144 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:03.590133 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f6f4130-52fa-46dc-a127-641a23eddf59-kube-api-access-k2hp7" (OuterVolumeSpecName: "kube-api-access-k2hp7") pod "4f6f4130-52fa-46dc-a127-641a23eddf59" (UID: "4f6f4130-52fa-46dc-a127-641a23eddf59"). InnerVolumeSpecName "kube-api-access-k2hp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:39:03.688858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:03.688817 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2hp7\" (UniqueName: \"kubernetes.io/projected/4f6f4130-52fa-46dc-a127-641a23eddf59-kube-api-access-k2hp7\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:39:03.688858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:03.688857 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f6f4130-52fa-46dc-a127-641a23eddf59-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:39:03.688858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:03.688867 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f6f4130-52fa-46dc-a127-641a23eddf59-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:39:04.345044 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:04.345007 2573 generic.go:356] "Generic (PLEG): container finished" podID="4f6f4130-52fa-46dc-a127-641a23eddf59" containerID="a1803a98dd97b1c10d76ba0f2545362ba08f60c323183c25554763e0e15d5e66" exitCode=0 Jul 03 10:39:04.345453 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:04.345053 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" Jul 03 10:39:04.345453 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:04.345095 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" event={"ID":"4f6f4130-52fa-46dc-a127-641a23eddf59","Type":"ContainerDied","Data":"a1803a98dd97b1c10d76ba0f2545362ba08f60c323183c25554763e0e15d5e66"} Jul 03 10:39:04.345453 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:04.345148 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b" event={"ID":"4f6f4130-52fa-46dc-a127-641a23eddf59","Type":"ContainerDied","Data":"44f2fa21eb07d5f32aa61e71b918bb8b4151ce001ea977153c814760303bc608"} Jul 03 10:39:04.345453 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:04.345178 2573 scope.go:117] "RemoveContainer" containerID="a1803a98dd97b1c10d76ba0f2545362ba08f60c323183c25554763e0e15d5e66" Jul 03 10:39:04.359093 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:04.359071 2573 scope.go:117] "RemoveContainer" containerID="a1803a98dd97b1c10d76ba0f2545362ba08f60c323183c25554763e0e15d5e66" Jul 03 10:39:04.359503 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:39:04.359459 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1803a98dd97b1c10d76ba0f2545362ba08f60c323183c25554763e0e15d5e66\": container with ID starting with a1803a98dd97b1c10d76ba0f2545362ba08f60c323183c25554763e0e15d5e66 not found: ID does not exist" containerID="a1803a98dd97b1c10d76ba0f2545362ba08f60c323183c25554763e0e15d5e66" Jul 03 10:39:04.359566 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:04.359526 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1803a98dd97b1c10d76ba0f2545362ba08f60c323183c25554763e0e15d5e66"} err="failed to get container status \"a1803a98dd97b1c10d76ba0f2545362ba08f60c323183c25554763e0e15d5e66\": rpc error: code = NotFound desc = could not find container \"a1803a98dd97b1c10d76ba0f2545362ba08f60c323183c25554763e0e15d5e66\": container with ID starting with a1803a98dd97b1c10d76ba0f2545362ba08f60c323183c25554763e0e15d5e66 not found: ID does not exist" Jul 03 10:39:04.376559 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:04.376532 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b"] Jul 03 10:39:04.382968 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:04.382946 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-7436d1-5548866c7f-zhf6b"] Jul 03 10:39:04.563629 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:04.563596 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f6f4130-52fa-46dc-a127-641a23eddf59" path="/var/lib/kubelet/pods/4f6f4130-52fa-46dc-a127-641a23eddf59/volumes" Jul 03 10:39:14.285176 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:14.285141 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht"] Jul 03 10:39:14.285581 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:14.285537 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36579fb9-7857-493a-9d88-e7b178824735" containerName="main" Jul 03 10:39:14.285581 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:14.285549 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="36579fb9-7857-493a-9d88-e7b178824735" containerName="main" Jul 03 10:39:14.285581 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:14.285565 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f6f4130-52fa-46dc-a127-641a23eddf59" containerName="maas-api" Jul 03 10:39:14.285581 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:14.285571 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f6f4130-52fa-46dc-a127-641a23eddf59" containerName="maas-api" Jul 03 10:39:14.285727 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:14.285636 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f6f4130-52fa-46dc-a127-641a23eddf59" containerName="maas-api" Jul 03 10:39:14.285727 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:14.285645 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="36579fb9-7857-493a-9d88-e7b178824735" containerName="main" Jul 03 10:39:14.290305 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:14.290286 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" Jul 03 10:39:14.294895 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:14.294872 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-kc4sf\"" Jul 03 10:39:14.295118 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:14.295105 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-b8b776-serving-cert\"" Jul 03 10:39:14.306658 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:14.306634 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht"] Jul 03 10:39:14.393708 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:14.393671 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b18f4aa8-c4e0-4545-95be-959f4af6b90b-maas-api-tls\") pod \"maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht\" (UID: \"b18f4aa8-c4e0-4545-95be-959f4af6b90b\") " pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" Jul 03 10:39:14.393893 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:14.393723 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j45vc\" (UniqueName: \"kubernetes.io/projected/b18f4aa8-c4e0-4545-95be-959f4af6b90b-kube-api-access-j45vc\") pod \"maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht\" (UID: \"b18f4aa8-c4e0-4545-95be-959f4af6b90b\") " pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" Jul 03 10:39:14.393893 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:14.393872 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b18f4aa8-c4e0-4545-95be-959f4af6b90b-openshift-service-ca\") pod \"maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht\" (UID: \"b18f4aa8-c4e0-4545-95be-959f4af6b90b\") " pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" Jul 03 10:39:14.495316 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:14.495280 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b18f4aa8-c4e0-4545-95be-959f4af6b90b-openshift-service-ca\") pod \"maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht\" (UID: \"b18f4aa8-c4e0-4545-95be-959f4af6b90b\") " pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" Jul 03 10:39:14.495503 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:14.495336 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b18f4aa8-c4e0-4545-95be-959f4af6b90b-maas-api-tls\") pod \"maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht\" (UID: \"b18f4aa8-c4e0-4545-95be-959f4af6b90b\") " pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" Jul 03 10:39:14.495503 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:14.495368 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j45vc\" (UniqueName: \"kubernetes.io/projected/b18f4aa8-c4e0-4545-95be-959f4af6b90b-kube-api-access-j45vc\") pod \"maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht\" (UID: \"b18f4aa8-c4e0-4545-95be-959f4af6b90b\") " pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" Jul 03 10:39:14.495634 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:39:14.495512 2573 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-b8b776-serving-cert: secret "maas-api-e2e-api-a-b8b776-serving-cert" not found Jul 03 10:39:14.495634 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:39:14.495600 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b18f4aa8-c4e0-4545-95be-959f4af6b90b-maas-api-tls podName:b18f4aa8-c4e0-4545-95be-959f4af6b90b nodeName:}" failed. No retries permitted until 2026-07-03 10:39:14.995577224 +0000 UTC m=+2009.907466523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b18f4aa8-c4e0-4545-95be-959f4af6b90b-maas-api-tls") pod "maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" (UID: "b18f4aa8-c4e0-4545-95be-959f4af6b90b") : secret "maas-api-e2e-api-a-b8b776-serving-cert" not found Jul 03 10:39:14.495986 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:14.495965 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b18f4aa8-c4e0-4545-95be-959f4af6b90b-openshift-service-ca\") pod \"maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht\" (UID: \"b18f4aa8-c4e0-4545-95be-959f4af6b90b\") " pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" Jul 03 10:39:14.515687 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:14.515652 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j45vc\" (UniqueName: \"kubernetes.io/projected/b18f4aa8-c4e0-4545-95be-959f4af6b90b-kube-api-access-j45vc\") pod \"maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht\" (UID: \"b18f4aa8-c4e0-4545-95be-959f4af6b90b\") " pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" Jul 03 10:39:14.999446 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:14.999414 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b18f4aa8-c4e0-4545-95be-959f4af6b90b-maas-api-tls\") pod \"maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht\" (UID: \"b18f4aa8-c4e0-4545-95be-959f4af6b90b\") " pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" Jul 03 10:39:15.001829 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:15.001801 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b18f4aa8-c4e0-4545-95be-959f4af6b90b-maas-api-tls\") pod \"maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht\" (UID: \"b18f4aa8-c4e0-4545-95be-959f4af6b90b\") " pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" Jul 03 10:39:15.200052 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:15.200013 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" Jul 03 10:39:15.333265 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:15.333240 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht"] Jul 03 10:39:15.335484 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:39:15.335457 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb18f4aa8_c4e0_4545_95be_959f4af6b90b.slice/crio-66d8fe221dabd9bf46282368ad3d7957fae6bf92e7a37cee0c6e98ec5744007f WatchSource:0}: Error finding container 66d8fe221dabd9bf46282368ad3d7957fae6bf92e7a37cee0c6e98ec5744007f: Status 404 returned error can't find the container with id 66d8fe221dabd9bf46282368ad3d7957fae6bf92e7a37cee0c6e98ec5744007f Jul 03 10:39:15.404075 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:15.404040 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" event={"ID":"b18f4aa8-c4e0-4545-95be-959f4af6b90b","Type":"ContainerStarted","Data":"66d8fe221dabd9bf46282368ad3d7957fae6bf92e7a37cee0c6e98ec5744007f"} Jul 03 10:39:16.410601 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:16.410558 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" event={"ID":"b18f4aa8-c4e0-4545-95be-959f4af6b90b","Type":"ContainerStarted","Data":"2ada2aca1b18f36c11eef16f11822d1411416dfe698e43329d3c1c293966c142"} Jul 03 10:39:16.411042 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:16.410768 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" Jul 03 10:39:16.430715 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:16.430658 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" podStartSLOduration=1.922779097 podStartE2EDuration="2.430641481s" totalImagesPullingTime="507.862384ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:39:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:39:16.428954644 +0000 UTC m=+2011.340843952" watchObservedRunningTime="2026-07-03 10:39:16.430641481 +0000 UTC m=+2011.342530785" Jul 03 10:39:22.167349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:22.167311 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt"] Jul 03 10:39:22.171295 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:22.171272 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" Jul 03 10:39:22.174858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:22.174832 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-84ecdb-serving-cert\"" Jul 03 10:39:22.185070 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:22.185045 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt"] Jul 03 10:39:22.269545 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:22.269515 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4699705-9d3a-4e29-a358-f925b45bc035-maas-api-tls\") pod \"maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt\" (UID: \"f4699705-9d3a-4e29-a358-f925b45bc035\") " pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" Jul 03 10:39:22.269713 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:22.269547 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4699705-9d3a-4e29-a358-f925b45bc035-openshift-service-ca\") pod \"maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt\" (UID: \"f4699705-9d3a-4e29-a358-f925b45bc035\") " pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" Jul 03 10:39:22.269713 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:22.269593 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4kmn\" (UniqueName: \"kubernetes.io/projected/f4699705-9d3a-4e29-a358-f925b45bc035-kube-api-access-x4kmn\") pod \"maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt\" (UID: \"f4699705-9d3a-4e29-a358-f925b45bc035\") " pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" Jul 03 10:39:22.371128 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:22.371077 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4699705-9d3a-4e29-a358-f925b45bc035-maas-api-tls\") pod \"maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt\" (UID: \"f4699705-9d3a-4e29-a358-f925b45bc035\") " pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" Jul 03 10:39:22.371128 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:22.371131 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4699705-9d3a-4e29-a358-f925b45bc035-openshift-service-ca\") pod \"maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt\" (UID: \"f4699705-9d3a-4e29-a358-f925b45bc035\") " pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" Jul 03 10:39:22.371367 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:22.371187 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4kmn\" (UniqueName: \"kubernetes.io/projected/f4699705-9d3a-4e29-a358-f925b45bc035-kube-api-access-x4kmn\") pod \"maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt\" (UID: \"f4699705-9d3a-4e29-a358-f925b45bc035\") " pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" Jul 03 10:39:22.371367 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:39:22.371243 2573 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-84ecdb-serving-cert: secret "maas-api-e2e-api-b-84ecdb-serving-cert" not found Jul 03 10:39:22.371367 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:39:22.371328 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4699705-9d3a-4e29-a358-f925b45bc035-maas-api-tls podName:f4699705-9d3a-4e29-a358-f925b45bc035 nodeName:}" failed. No retries permitted until 2026-07-03 10:39:22.871305308 +0000 UTC m=+2017.783194595 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f4699705-9d3a-4e29-a358-f925b45bc035-maas-api-tls") pod "maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" (UID: "f4699705-9d3a-4e29-a358-f925b45bc035") : secret "maas-api-e2e-api-b-84ecdb-serving-cert" not found Jul 03 10:39:22.371817 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:22.371798 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4699705-9d3a-4e29-a358-f925b45bc035-openshift-service-ca\") pod \"maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt\" (UID: \"f4699705-9d3a-4e29-a358-f925b45bc035\") " pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" Jul 03 10:39:22.382065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:22.382041 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4kmn\" (UniqueName: \"kubernetes.io/projected/f4699705-9d3a-4e29-a358-f925b45bc035-kube-api-access-x4kmn\") pod \"maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt\" (UID: \"f4699705-9d3a-4e29-a358-f925b45bc035\") " pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" Jul 03 10:39:22.422006 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:22.421931 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" Jul 03 10:39:22.876163 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:22.876077 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4699705-9d3a-4e29-a358-f925b45bc035-maas-api-tls\") pod \"maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt\" (UID: \"f4699705-9d3a-4e29-a358-f925b45bc035\") " pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" Jul 03 10:39:22.878512 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:22.878485 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4699705-9d3a-4e29-a358-f925b45bc035-maas-api-tls\") pod \"maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt\" (UID: \"f4699705-9d3a-4e29-a358-f925b45bc035\") " pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" Jul 03 10:39:23.083926 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:23.083886 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" Jul 03 10:39:23.224515 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:23.224487 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt"] Jul 03 10:39:23.226201 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:39:23.226169 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4699705_9d3a_4e29_a358_f925b45bc035.slice/crio-5c971cf179c0f13dcf838ec160016ff6f494672337b42dd25255aad310106ff9 WatchSource:0}: Error finding container 5c971cf179c0f13dcf838ec160016ff6f494672337b42dd25255aad310106ff9: Status 404 returned error can't find the container with id 5c971cf179c0f13dcf838ec160016ff6f494672337b42dd25255aad310106ff9 Jul 03 10:39:23.447222 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:23.447132 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" event={"ID":"f4699705-9d3a-4e29-a358-f925b45bc035","Type":"ContainerStarted","Data":"5c971cf179c0f13dcf838ec160016ff6f494672337b42dd25255aad310106ff9"} Jul 03 10:39:24.453147 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:24.453112 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" event={"ID":"f4699705-9d3a-4e29-a358-f925b45bc035","Type":"ContainerStarted","Data":"56188aa94e1454f00fcbbbea8324e9eed2fb8c4436b4d9a28bf8c7b5e06cbde1"} Jul 03 10:39:24.453581 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:24.453206 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" Jul 03 10:39:24.475447 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:24.475386 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" podStartSLOduration=1.8711821579999999 podStartE2EDuration="2.47536686s" totalImagesPullingTime="604.184702ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:39:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:39:24.474901195 +0000 UTC m=+2019.386790500" watchObservedRunningTime="2026-07-03 10:39:24.47536686 +0000 UTC m=+2019.387256164" Jul 03 10:39:30.462767 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:30.462722 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" Jul 03 10:39:35.604714 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:35.604677 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht"] Jul 03 10:39:35.605254 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:35.604948 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" podUID="b18f4aa8-c4e0-4545-95be-959f4af6b90b" containerName="maas-api" containerID="cri-o://2ada2aca1b18f36c11eef16f11822d1411416dfe698e43329d3c1c293966c142" gracePeriod=30 Jul 03 10:39:35.852550 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:35.852524 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" Jul 03 10:39:36.004948 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:36.004917 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j45vc\" (UniqueName: \"kubernetes.io/projected/b18f4aa8-c4e0-4545-95be-959f4af6b90b-kube-api-access-j45vc\") pod \"b18f4aa8-c4e0-4545-95be-959f4af6b90b\" (UID: \"b18f4aa8-c4e0-4545-95be-959f4af6b90b\") " Jul 03 10:39:36.005131 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:36.005019 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b18f4aa8-c4e0-4545-95be-959f4af6b90b-maas-api-tls\") pod \"b18f4aa8-c4e0-4545-95be-959f4af6b90b\" (UID: \"b18f4aa8-c4e0-4545-95be-959f4af6b90b\") " Jul 03 10:39:36.005131 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:36.005056 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b18f4aa8-c4e0-4545-95be-959f4af6b90b-openshift-service-ca\") pod \"b18f4aa8-c4e0-4545-95be-959f4af6b90b\" (UID: \"b18f4aa8-c4e0-4545-95be-959f4af6b90b\") " Jul 03 10:39:36.005452 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:36.005430 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b18f4aa8-c4e0-4545-95be-959f4af6b90b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b18f4aa8-c4e0-4545-95be-959f4af6b90b" (UID: "b18f4aa8-c4e0-4545-95be-959f4af6b90b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:39:36.006995 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:36.006968 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b18f4aa8-c4e0-4545-95be-959f4af6b90b-kube-api-access-j45vc" (OuterVolumeSpecName: "kube-api-access-j45vc") pod "b18f4aa8-c4e0-4545-95be-959f4af6b90b" (UID: "b18f4aa8-c4e0-4545-95be-959f4af6b90b"). InnerVolumeSpecName "kube-api-access-j45vc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:39:36.007179 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:36.007157 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b18f4aa8-c4e0-4545-95be-959f4af6b90b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b18f4aa8-c4e0-4545-95be-959f4af6b90b" (UID: "b18f4aa8-c4e0-4545-95be-959f4af6b90b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:39:36.106241 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:36.106195 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j45vc\" (UniqueName: \"kubernetes.io/projected/b18f4aa8-c4e0-4545-95be-959f4af6b90b-kube-api-access-j45vc\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:39:36.106241 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:36.106231 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b18f4aa8-c4e0-4545-95be-959f4af6b90b-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:39:36.106241 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:36.106244 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b18f4aa8-c4e0-4545-95be-959f4af6b90b-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:39:36.515549 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:36.515517 2573 generic.go:356] "Generic (PLEG): container finished" podID="b18f4aa8-c4e0-4545-95be-959f4af6b90b" containerID="2ada2aca1b18f36c11eef16f11822d1411416dfe698e43329d3c1c293966c142" exitCode=0 Jul 03 10:39:36.515785 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:36.515569 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" Jul 03 10:39:36.515785 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:36.515606 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" event={"ID":"b18f4aa8-c4e0-4545-95be-959f4af6b90b","Type":"ContainerDied","Data":"2ada2aca1b18f36c11eef16f11822d1411416dfe698e43329d3c1c293966c142"} Jul 03 10:39:36.515785 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:36.515661 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht" event={"ID":"b18f4aa8-c4e0-4545-95be-959f4af6b90b","Type":"ContainerDied","Data":"66d8fe221dabd9bf46282368ad3d7957fae6bf92e7a37cee0c6e98ec5744007f"} Jul 03 10:39:36.515785 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:36.515683 2573 scope.go:117] "RemoveContainer" containerID="2ada2aca1b18f36c11eef16f11822d1411416dfe698e43329d3c1c293966c142" Jul 03 10:39:36.524474 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:36.524457 2573 scope.go:117] "RemoveContainer" containerID="2ada2aca1b18f36c11eef16f11822d1411416dfe698e43329d3c1c293966c142" Jul 03 10:39:36.524721 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:39:36.524699 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ada2aca1b18f36c11eef16f11822d1411416dfe698e43329d3c1c293966c142\": container with ID starting with 2ada2aca1b18f36c11eef16f11822d1411416dfe698e43329d3c1c293966c142 not found: ID does not exist" containerID="2ada2aca1b18f36c11eef16f11822d1411416dfe698e43329d3c1c293966c142" Jul 03 10:39:36.524810 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:36.524750 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ada2aca1b18f36c11eef16f11822d1411416dfe698e43329d3c1c293966c142"} err="failed to get container status \"2ada2aca1b18f36c11eef16f11822d1411416dfe698e43329d3c1c293966c142\": rpc error: code = NotFound desc = could not find container \"2ada2aca1b18f36c11eef16f11822d1411416dfe698e43329d3c1c293966c142\": container with ID starting with 2ada2aca1b18f36c11eef16f11822d1411416dfe698e43329d3c1c293966c142 not found: ID does not exist" Jul 03 10:39:36.539204 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:36.539178 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht"] Jul 03 10:39:36.543685 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:36.543660 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-b8b776-8df86b66b-wr9ht"] Jul 03 10:39:36.563997 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:36.563972 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b18f4aa8-c4e0-4545-95be-959f4af6b90b" path="/var/lib/kubelet/pods/b18f4aa8-c4e0-4545-95be-959f4af6b90b/volumes" Jul 03 10:39:42.715469 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:42.715438 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt"] Jul 03 10:39:42.715879 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:42.715662 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" podUID="f4699705-9d3a-4e29-a358-f925b45bc035" containerName="maas-api" containerID="cri-o://56188aa94e1454f00fcbbbea8324e9eed2fb8c4436b4d9a28bf8c7b5e06cbde1" gracePeriod=30 Jul 03 10:39:42.959158 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:42.959131 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" Jul 03 10:39:43.065007 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:43.064921 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4699705-9d3a-4e29-a358-f925b45bc035-maas-api-tls\") pod \"f4699705-9d3a-4e29-a358-f925b45bc035\" (UID: \"f4699705-9d3a-4e29-a358-f925b45bc035\") " Jul 03 10:39:43.065176 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:43.065016 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4kmn\" (UniqueName: \"kubernetes.io/projected/f4699705-9d3a-4e29-a358-f925b45bc035-kube-api-access-x4kmn\") pod \"f4699705-9d3a-4e29-a358-f925b45bc035\" (UID: \"f4699705-9d3a-4e29-a358-f925b45bc035\") " Jul 03 10:39:43.065176 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:43.065071 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4699705-9d3a-4e29-a358-f925b45bc035-openshift-service-ca\") pod \"f4699705-9d3a-4e29-a358-f925b45bc035\" (UID: \"f4699705-9d3a-4e29-a358-f925b45bc035\") " Jul 03 10:39:43.065487 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:43.065458 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4699705-9d3a-4e29-a358-f925b45bc035-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f4699705-9d3a-4e29-a358-f925b45bc035" (UID: "f4699705-9d3a-4e29-a358-f925b45bc035"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:39:43.066936 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:43.066910 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4699705-9d3a-4e29-a358-f925b45bc035-kube-api-access-x4kmn" (OuterVolumeSpecName: "kube-api-access-x4kmn") pod "f4699705-9d3a-4e29-a358-f925b45bc035" (UID: "f4699705-9d3a-4e29-a358-f925b45bc035"). InnerVolumeSpecName "kube-api-access-x4kmn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:39:43.067026 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:43.066937 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4699705-9d3a-4e29-a358-f925b45bc035-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f4699705-9d3a-4e29-a358-f925b45bc035" (UID: "f4699705-9d3a-4e29-a358-f925b45bc035"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:39:43.165797 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:43.165727 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4699705-9d3a-4e29-a358-f925b45bc035-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:39:43.165797 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:43.165792 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f4699705-9d3a-4e29-a358-f925b45bc035-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:39:43.165797 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:43.165802 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4kmn\" (UniqueName: \"kubernetes.io/projected/f4699705-9d3a-4e29-a358-f925b45bc035-kube-api-access-x4kmn\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:39:43.552567 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:43.552530 2573 generic.go:356] "Generic (PLEG): container finished" podID="f4699705-9d3a-4e29-a358-f925b45bc035" containerID="56188aa94e1454f00fcbbbea8324e9eed2fb8c4436b4d9a28bf8c7b5e06cbde1" exitCode=0 Jul 03 10:39:43.552798 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:43.552572 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" Jul 03 10:39:43.552798 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:43.552639 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" event={"ID":"f4699705-9d3a-4e29-a358-f925b45bc035","Type":"ContainerDied","Data":"56188aa94e1454f00fcbbbea8324e9eed2fb8c4436b4d9a28bf8c7b5e06cbde1"} Jul 03 10:39:43.552798 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:43.552688 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt" event={"ID":"f4699705-9d3a-4e29-a358-f925b45bc035","Type":"ContainerDied","Data":"5c971cf179c0f13dcf838ec160016ff6f494672337b42dd25255aad310106ff9"} Jul 03 10:39:43.552798 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:43.552712 2573 scope.go:117] "RemoveContainer" containerID="56188aa94e1454f00fcbbbea8324e9eed2fb8c4436b4d9a28bf8c7b5e06cbde1" Jul 03 10:39:43.561422 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:43.561405 2573 scope.go:117] "RemoveContainer" containerID="56188aa94e1454f00fcbbbea8324e9eed2fb8c4436b4d9a28bf8c7b5e06cbde1" Jul 03 10:39:43.561661 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:39:43.561642 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56188aa94e1454f00fcbbbea8324e9eed2fb8c4436b4d9a28bf8c7b5e06cbde1\": container with ID starting with 56188aa94e1454f00fcbbbea8324e9eed2fb8c4436b4d9a28bf8c7b5e06cbde1 not found: ID does not exist" containerID="56188aa94e1454f00fcbbbea8324e9eed2fb8c4436b4d9a28bf8c7b5e06cbde1" Jul 03 10:39:43.561705 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:43.561673 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56188aa94e1454f00fcbbbea8324e9eed2fb8c4436b4d9a28bf8c7b5e06cbde1"} err="failed to get container status \"56188aa94e1454f00fcbbbea8324e9eed2fb8c4436b4d9a28bf8c7b5e06cbde1\": rpc error: code = NotFound desc = could not find container \"56188aa94e1454f00fcbbbea8324e9eed2fb8c4436b4d9a28bf8c7b5e06cbde1\": container with ID starting with 56188aa94e1454f00fcbbbea8324e9eed2fb8c4436b4d9a28bf8c7b5e06cbde1 not found: ID does not exist" Jul 03 10:39:43.583293 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:43.583264 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt"] Jul 03 10:39:43.586475 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:43.586454 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-84ecdb-6575bd44bf-bnnvt"] Jul 03 10:39:44.562557 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:44.562521 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4699705-9d3a-4e29-a358-f925b45bc035" path="/var/lib/kubelet/pods/f4699705-9d3a-4e29-a358-f925b45bc035/volumes" Jul 03 10:39:52.230321 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.230284 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd"] Jul 03 10:39:52.230929 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.230911 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b18f4aa8-c4e0-4545-95be-959f4af6b90b" containerName="maas-api" Jul 03 10:39:52.230986 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.230933 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b18f4aa8-c4e0-4545-95be-959f4af6b90b" containerName="maas-api" Jul 03 10:39:52.230986 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.230957 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4699705-9d3a-4e29-a358-f925b45bc035" containerName="maas-api" Jul 03 10:39:52.230986 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.230966 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4699705-9d3a-4e29-a358-f925b45bc035" containerName="maas-api" Jul 03 10:39:52.231097 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.231074 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b18f4aa8-c4e0-4545-95be-959f4af6b90b" containerName="maas-api" Jul 03 10:39:52.231097 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.231086 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4699705-9d3a-4e29-a358-f925b45bc035" containerName="maas-api" Jul 03 10:39:52.234473 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.234452 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" Jul 03 10:39:52.237264 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.237244 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-cda536-serving-cert\"" Jul 03 10:39:52.237400 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.237323 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-kc4sf\"" Jul 03 10:39:52.247254 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.247226 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd"] Jul 03 10:39:52.349280 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.349238 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49klz\" (UniqueName: \"kubernetes.io/projected/569893b6-194a-41e3-8bd0-9563864ea7ea-kube-api-access-49klz\") pod \"maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd\" (UID: \"569893b6-194a-41e3-8bd0-9563864ea7ea\") " pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" Jul 03 10:39:52.349475 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.349290 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/569893b6-194a-41e3-8bd0-9563864ea7ea-maas-api-tls\") pod \"maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd\" (UID: \"569893b6-194a-41e3-8bd0-9563864ea7ea\") " pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" Jul 03 10:39:52.349475 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.349357 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/569893b6-194a-41e3-8bd0-9563864ea7ea-openshift-service-ca\") pod \"maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd\" (UID: \"569893b6-194a-41e3-8bd0-9563864ea7ea\") " pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" Jul 03 10:39:52.450637 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.450589 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/569893b6-194a-41e3-8bd0-9563864ea7ea-openshift-service-ca\") pod \"maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd\" (UID: \"569893b6-194a-41e3-8bd0-9563864ea7ea\") " pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" Jul 03 10:39:52.450812 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.450714 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49klz\" (UniqueName: \"kubernetes.io/projected/569893b6-194a-41e3-8bd0-9563864ea7ea-kube-api-access-49klz\") pod \"maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd\" (UID: \"569893b6-194a-41e3-8bd0-9563864ea7ea\") " pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" Jul 03 10:39:52.450812 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.450792 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/569893b6-194a-41e3-8bd0-9563864ea7ea-maas-api-tls\") pod \"maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd\" (UID: \"569893b6-194a-41e3-8bd0-9563864ea7ea\") " pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" Jul 03 10:39:52.450965 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:39:52.450941 2573 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-cda536-serving-cert: secret "maas-api-e2e-shared-a-cda536-serving-cert" not found Jul 03 10:39:52.451035 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:39:52.451024 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/569893b6-194a-41e3-8bd0-9563864ea7ea-maas-api-tls podName:569893b6-194a-41e3-8bd0-9563864ea7ea nodeName:}" failed. No retries permitted until 2026-07-03 10:39:52.951005849 +0000 UTC m=+2047.862895135 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/569893b6-194a-41e3-8bd0-9563864ea7ea-maas-api-tls") pod "maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" (UID: "569893b6-194a-41e3-8bd0-9563864ea7ea") : secret "maas-api-e2e-shared-a-cda536-serving-cert" not found Jul 03 10:39:52.451344 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.451319 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/569893b6-194a-41e3-8bd0-9563864ea7ea-openshift-service-ca\") pod \"maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd\" (UID: \"569893b6-194a-41e3-8bd0-9563864ea7ea\") " pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" Jul 03 10:39:52.473247 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.473205 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49klz\" (UniqueName: \"kubernetes.io/projected/569893b6-194a-41e3-8bd0-9563864ea7ea-kube-api-access-49klz\") pod \"maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd\" (UID: \"569893b6-194a-41e3-8bd0-9563864ea7ea\") " pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" Jul 03 10:39:52.955170 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.955118 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/569893b6-194a-41e3-8bd0-9563864ea7ea-maas-api-tls\") pod \"maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd\" (UID: \"569893b6-194a-41e3-8bd0-9563864ea7ea\") " pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" Jul 03 10:39:52.957514 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:52.957485 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/569893b6-194a-41e3-8bd0-9563864ea7ea-maas-api-tls\") pod \"maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd\" (UID: \"569893b6-194a-41e3-8bd0-9563864ea7ea\") " pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" Jul 03 10:39:53.145937 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:53.145905 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" Jul 03 10:39:53.284996 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:53.284970 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd"] Jul 03 10:39:53.286904 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:39:53.286882 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod569893b6_194a_41e3_8bd0_9563864ea7ea.slice/crio-00ab1c9d40747756f2bc1fa3150423542eddd1c4ba07affa47e8e9d11d4abee7 WatchSource:0}: Error finding container 00ab1c9d40747756f2bc1fa3150423542eddd1c4ba07affa47e8e9d11d4abee7: Status 404 returned error can't find the container with id 00ab1c9d40747756f2bc1fa3150423542eddd1c4ba07affa47e8e9d11d4abee7 Jul 03 10:39:53.606391 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:53.606303 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" event={"ID":"569893b6-194a-41e3-8bd0-9563864ea7ea","Type":"ContainerStarted","Data":"00ab1c9d40747756f2bc1fa3150423542eddd1c4ba07affa47e8e9d11d4abee7"} Jul 03 10:39:54.612908 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:54.612872 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" event={"ID":"569893b6-194a-41e3-8bd0-9563864ea7ea","Type":"ContainerStarted","Data":"8b8a173fe80b29d80283f0606cb03f3ab4b9c7d58b40e8c044e2a00d5c3810bf"} Jul 03 10:39:54.613313 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:54.612942 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" Jul 03 10:39:54.633548 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:39:54.633494 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" podStartSLOduration=1.877526439 podStartE2EDuration="2.633479334s" totalImagesPullingTime="755.952895ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:39:54.630557155 +0000 UTC m=+2049.542446459" watchObservedRunningTime="2026-07-03 10:39:54.633479334 +0000 UTC m=+2049.545368638" Jul 03 10:40:00.280336 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:00.280299 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk"] Jul 03 10:40:00.284051 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:00.284029 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" Jul 03 10:40:00.287057 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:00.287024 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-62c45e-serving-cert\"" Jul 03 10:40:00.298832 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:00.298809 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk"] Jul 03 10:40:00.326953 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:00.326919 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9565699-e375-4d15-b61d-82d015b96298-maas-api-tls\") pod \"maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk\" (UID: \"c9565699-e375-4d15-b61d-82d015b96298\") " pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" Jul 03 10:40:00.327107 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:00.326959 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9565699-e375-4d15-b61d-82d015b96298-openshift-service-ca\") pod \"maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk\" (UID: \"c9565699-e375-4d15-b61d-82d015b96298\") " pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" Jul 03 10:40:00.327107 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:00.327012 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t52qc\" (UniqueName: \"kubernetes.io/projected/c9565699-e375-4d15-b61d-82d015b96298-kube-api-access-t52qc\") pod \"maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk\" (UID: \"c9565699-e375-4d15-b61d-82d015b96298\") " pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" Jul 03 10:40:00.427578 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:00.427546 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t52qc\" (UniqueName: \"kubernetes.io/projected/c9565699-e375-4d15-b61d-82d015b96298-kube-api-access-t52qc\") pod \"maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk\" (UID: \"c9565699-e375-4d15-b61d-82d015b96298\") " pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" Jul 03 10:40:00.427759 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:00.427634 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9565699-e375-4d15-b61d-82d015b96298-maas-api-tls\") pod \"maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk\" (UID: \"c9565699-e375-4d15-b61d-82d015b96298\") " pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" Jul 03 10:40:00.427759 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:00.427656 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9565699-e375-4d15-b61d-82d015b96298-openshift-service-ca\") pod \"maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk\" (UID: \"c9565699-e375-4d15-b61d-82d015b96298\") " pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" Jul 03 10:40:00.427853 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:40:00.427771 2573 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-62c45e-serving-cert: secret "maas-api-e2e-shared-b-62c45e-serving-cert" not found Jul 03 10:40:00.427853 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:40:00.427831 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9565699-e375-4d15-b61d-82d015b96298-maas-api-tls podName:c9565699-e375-4d15-b61d-82d015b96298 nodeName:}" failed. No retries permitted until 2026-07-03 10:40:00.927813868 +0000 UTC m=+2055.839703150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c9565699-e375-4d15-b61d-82d015b96298-maas-api-tls") pod "maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" (UID: "c9565699-e375-4d15-b61d-82d015b96298") : secret "maas-api-e2e-shared-b-62c45e-serving-cert" not found Jul 03 10:40:00.428243 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:00.428227 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9565699-e375-4d15-b61d-82d015b96298-openshift-service-ca\") pod \"maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk\" (UID: \"c9565699-e375-4d15-b61d-82d015b96298\") " pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" Jul 03 10:40:00.437836 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:00.437811 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t52qc\" (UniqueName: \"kubernetes.io/projected/c9565699-e375-4d15-b61d-82d015b96298-kube-api-access-t52qc\") pod \"maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk\" (UID: \"c9565699-e375-4d15-b61d-82d015b96298\") " pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" Jul 03 10:40:00.624409 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:00.624328 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" Jul 03 10:40:00.931804 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:00.931761 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9565699-e375-4d15-b61d-82d015b96298-maas-api-tls\") pod \"maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk\" (UID: \"c9565699-e375-4d15-b61d-82d015b96298\") " pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" Jul 03 10:40:00.934198 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:00.934168 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9565699-e375-4d15-b61d-82d015b96298-maas-api-tls\") pod \"maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk\" (UID: \"c9565699-e375-4d15-b61d-82d015b96298\") " pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" Jul 03 10:40:01.195323 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:01.195223 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" Jul 03 10:40:01.340706 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:01.340629 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk"] Jul 03 10:40:01.342697 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:40:01.342667 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9565699_e375_4d15_b61d_82d015b96298.slice/crio-301eeb9087d5b42e327d24dc050bfbeb1e20625618bc1381424f2e5ce0739ed6 WatchSource:0}: Error finding container 301eeb9087d5b42e327d24dc050bfbeb1e20625618bc1381424f2e5ce0739ed6: Status 404 returned error can't find the container with id 301eeb9087d5b42e327d24dc050bfbeb1e20625618bc1381424f2e5ce0739ed6 Jul 03 10:40:01.645385 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:01.645350 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" event={"ID":"c9565699-e375-4d15-b61d-82d015b96298","Type":"ContainerStarted","Data":"301eeb9087d5b42e327d24dc050bfbeb1e20625618bc1381424f2e5ce0739ed6"} Jul 03 10:40:02.652150 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:02.652113 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" event={"ID":"c9565699-e375-4d15-b61d-82d015b96298","Type":"ContainerStarted","Data":"5305a66390d8facbd26ce094250a124446b1161bedba03fe66a03e5b2bdd781e"} Jul 03 10:40:02.652605 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:02.652234 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" Jul 03 10:40:02.671553 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:02.671498 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" podStartSLOduration=2.291583315 podStartE2EDuration="2.671477737s" totalImagesPullingTime="379.894422ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:40:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:40:02.670694468 +0000 UTC m=+2057.582583766" watchObservedRunningTime="2026-07-03 10:40:02.671477737 +0000 UTC m=+2057.583367069" Jul 03 10:40:08.662422 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:08.662346 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" Jul 03 10:40:18.407220 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.407182 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49"] Jul 03 10:40:18.410928 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.410913 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:18.416842 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.416814 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-cda536\"/\"kube-root-ca.crt\"" Jul 03 10:40:18.417009 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.416851 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-cda536\"/\"openshift-service-ca.crt\"" Jul 03 10:40:18.417009 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.416816 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-cda536\"/\"auth-test-model-cda536-kserve-self-signed-certs\"" Jul 03 10:40:18.417009 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.416819 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-cda536\"/\"default-dockercfg-948f4\"" Jul 03 10:40:18.430806 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.430777 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49"] Jul 03 10:40:18.502239 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.502196 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84ebf521-8154-447b-b367-443f7dff3dfa-tls-certs\") pod \"auth-test-model-cda536-kserve-5b57449ff9-7dz49\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:18.502239 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.502242 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-dshm\") pod \"auth-test-model-cda536-kserve-5b57449ff9-7dz49\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:18.502492 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.502324 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4gwc\" (UniqueName: \"kubernetes.io/projected/84ebf521-8154-447b-b367-443f7dff3dfa-kube-api-access-v4gwc\") pod \"auth-test-model-cda536-kserve-5b57449ff9-7dz49\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:18.502492 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.502384 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-model-cache\") pod \"auth-test-model-cda536-kserve-5b57449ff9-7dz49\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:18.502492 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.502438 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-home\") pod \"auth-test-model-cda536-kserve-5b57449ff9-7dz49\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:18.502605 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.502530 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-tmp-dir\") pod \"auth-test-model-cda536-kserve-5b57449ff9-7dz49\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:18.603402 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.603364 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4gwc\" (UniqueName: \"kubernetes.io/projected/84ebf521-8154-447b-b367-443f7dff3dfa-kube-api-access-v4gwc\") pod \"auth-test-model-cda536-kserve-5b57449ff9-7dz49\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:18.603575 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.603428 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-model-cache\") pod \"auth-test-model-cda536-kserve-5b57449ff9-7dz49\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:18.603575 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.603455 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-home\") pod \"auth-test-model-cda536-kserve-5b57449ff9-7dz49\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:18.603575 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.603529 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-tmp-dir\") pod \"auth-test-model-cda536-kserve-5b57449ff9-7dz49\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:18.603575 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.603562 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84ebf521-8154-447b-b367-443f7dff3dfa-tls-certs\") pod \"auth-test-model-cda536-kserve-5b57449ff9-7dz49\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:18.603829 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.603592 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-dshm\") pod \"auth-test-model-cda536-kserve-5b57449ff9-7dz49\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:18.603946 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.603926 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-model-cache\") pod \"auth-test-model-cda536-kserve-5b57449ff9-7dz49\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:18.604014 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.603982 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-tmp-dir\") pod \"auth-test-model-cda536-kserve-5b57449ff9-7dz49\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:18.604014 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.603991 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-home\") pod \"auth-test-model-cda536-kserve-5b57449ff9-7dz49\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:18.605792 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.605773 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-dshm\") pod \"auth-test-model-cda536-kserve-5b57449ff9-7dz49\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:18.606088 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.606069 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84ebf521-8154-447b-b367-443f7dff3dfa-tls-certs\") pod \"auth-test-model-cda536-kserve-5b57449ff9-7dz49\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:18.613248 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.613221 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4gwc\" (UniqueName: \"kubernetes.io/projected/84ebf521-8154-447b-b367-443f7dff3dfa-kube-api-access-v4gwc\") pod \"auth-test-model-cda536-kserve-5b57449ff9-7dz49\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:18.720706 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.720625 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:18.860122 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:18.860096 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49"] Jul 03 10:40:18.861518 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:40:18.861493 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84ebf521_8154_447b_b367_443f7dff3dfa.slice/crio-962b3f0400631ab01337d243f355c516905a0784940fe46f8afa68dcb27fbeed WatchSource:0}: Error finding container 962b3f0400631ab01337d243f355c516905a0784940fe46f8afa68dcb27fbeed: Status 404 returned error can't find the container with id 962b3f0400631ab01337d243f355c516905a0784940fe46f8afa68dcb27fbeed Jul 03 10:40:19.742076 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:19.742038 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" event={"ID":"84ebf521-8154-447b-b367-443f7dff3dfa","Type":"ContainerStarted","Data":"326a5cfc4448a151c317423ec7506b6a220f9249e7924ce98d48047a58d33939"} Jul 03 10:40:19.742493 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:19.742091 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" event={"ID":"84ebf521-8154-447b-b367-443f7dff3dfa","Type":"ContainerStarted","Data":"962b3f0400631ab01337d243f355c516905a0784940fe46f8afa68dcb27fbeed"} Jul 03 10:40:19.800693 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:19.800637 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" podStartSLOduration=1.800619072 podStartE2EDuration="1.800619072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:40:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:40:19.799982152 +0000 UTC m=+2074.711871455" watchObservedRunningTime="2026-07-03 10:40:19.800619072 +0000 UTC m=+2074.712508354" Jul 03 10:40:28.721085 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:28.721041 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:28.721492 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:28.721327 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:28.733604 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:28.733581 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:40:28.795440 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:40:28.795409 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:43:19.543864 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:19.543782 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49"] Jul 03 10:43:28.692526 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.692495 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8"] Jul 03 10:43:28.696008 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.695991 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:28.699946 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.699924 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-cda536\"/\"sub-test-model-cda536-kserve-self-signed-certs\"" Jul 03 10:43:28.715384 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.715357 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8"] Jul 03 10:43:28.811278 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.811240 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-dshm\") pod \"sub-test-model-cda536-kserve-789b4c4587-9sxt8\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:28.811498 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.811293 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qncx\" (UniqueName: \"kubernetes.io/projected/48091010-a5fd-4ea9-bff4-91d1f7cb0618-kube-api-access-9qncx\") pod \"sub-test-model-cda536-kserve-789b4c4587-9sxt8\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:28.811498 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.811321 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48091010-a5fd-4ea9-bff4-91d1f7cb0618-tls-certs\") pod \"sub-test-model-cda536-kserve-789b4c4587-9sxt8\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:28.811498 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.811384 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-tmp-dir\") pod \"sub-test-model-cda536-kserve-789b4c4587-9sxt8\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:28.811498 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.811464 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-model-cache\") pod \"sub-test-model-cda536-kserve-789b4c4587-9sxt8\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:28.811679 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.811518 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-home\") pod \"sub-test-model-cda536-kserve-789b4c4587-9sxt8\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:28.912774 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.912695 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qncx\" (UniqueName: \"kubernetes.io/projected/48091010-a5fd-4ea9-bff4-91d1f7cb0618-kube-api-access-9qncx\") pod \"sub-test-model-cda536-kserve-789b4c4587-9sxt8\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:28.912774 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.912777 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48091010-a5fd-4ea9-bff4-91d1f7cb0618-tls-certs\") pod \"sub-test-model-cda536-kserve-789b4c4587-9sxt8\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:28.913034 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.912806 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-tmp-dir\") pod \"sub-test-model-cda536-kserve-789b4c4587-9sxt8\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:28.913034 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.912947 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-model-cache\") pod \"sub-test-model-cda536-kserve-789b4c4587-9sxt8\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:28.913034 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.913011 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-home\") pod \"sub-test-model-cda536-kserve-789b4c4587-9sxt8\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:28.913200 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.913102 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-dshm\") pod \"sub-test-model-cda536-kserve-789b4c4587-9sxt8\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:28.913200 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.913174 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-tmp-dir\") pod \"sub-test-model-cda536-kserve-789b4c4587-9sxt8\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:28.913301 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.913256 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-model-cache\") pod \"sub-test-model-cda536-kserve-789b4c4587-9sxt8\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:28.913382 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.913359 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-home\") pod \"sub-test-model-cda536-kserve-789b4c4587-9sxt8\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:28.915162 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.915142 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-dshm\") pod \"sub-test-model-cda536-kserve-789b4c4587-9sxt8\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:28.915340 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.915322 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48091010-a5fd-4ea9-bff4-91d1f7cb0618-tls-certs\") pod \"sub-test-model-cda536-kserve-789b4c4587-9sxt8\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:28.922467 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:28.922437 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qncx\" (UniqueName: \"kubernetes.io/projected/48091010-a5fd-4ea9-bff4-91d1f7cb0618-kube-api-access-9qncx\") pod \"sub-test-model-cda536-kserve-789b4c4587-9sxt8\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:29.004613 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:29.004514 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:29.149314 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:29.149288 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8"] Jul 03 10:43:29.150892 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:43:29.150863 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48091010_a5fd_4ea9_bff4_91d1f7cb0618.slice/crio-bea9a0da16378159cbb07ee8cdba29b4d8394c29fb36a8763f3ab593df3ba47f WatchSource:0}: Error finding container bea9a0da16378159cbb07ee8cdba29b4d8394c29fb36a8763f3ab593df3ba47f: Status 404 returned error can't find the container with id bea9a0da16378159cbb07ee8cdba29b4d8394c29fb36a8763f3ab593df3ba47f Jul 03 10:43:29.723248 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:29.723209 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" event={"ID":"48091010-a5fd-4ea9-bff4-91d1f7cb0618","Type":"ContainerStarted","Data":"ee0dffd1e92194487a6856d83455edf2e69d3273581365752fc46fce3719822d"} Jul 03 10:43:29.723631 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:29.723256 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" event={"ID":"48091010-a5fd-4ea9-bff4-91d1f7cb0618","Type":"ContainerStarted","Data":"bea9a0da16378159cbb07ee8cdba29b4d8394c29fb36a8763f3ab593df3ba47f"} Jul 03 10:43:29.743560 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:29.743507 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" podStartSLOduration=1.743493041 podStartE2EDuration="1.743493041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:43:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:43:29.741704071 +0000 UTC m=+2264.653593406" watchObservedRunningTime="2026-07-03 10:43:29.743493041 +0000 UTC m=+2264.655382347" Jul 03 10:43:34.564058 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:34.564020 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" podUID="84ebf521-8154-447b-b367-443f7dff3dfa" containerName="main" containerID="cri-o://326a5cfc4448a151c317423ec7506b6a220f9249e7924ce98d48047a58d33939" gracePeriod=45 Jul 03 10:43:34.749913 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:34.749883 2573 generic.go:356] "Generic (PLEG): container finished" podID="84ebf521-8154-447b-b367-443f7dff3dfa" containerID="326a5cfc4448a151c317423ec7506b6a220f9249e7924ce98d48047a58d33939" exitCode=0 Jul 03 10:43:34.750118 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:34.749952 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" event={"ID":"84ebf521-8154-447b-b367-443f7dff3dfa","Type":"ContainerDied","Data":"326a5cfc4448a151c317423ec7506b6a220f9249e7924ce98d48047a58d33939"} Jul 03 10:43:34.819349 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:34.819291 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:43:34.975885 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:34.975849 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-model-cache\") pod \"84ebf521-8154-447b-b367-443f7dff3dfa\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " Jul 03 10:43:34.976077 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:34.975918 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-home\") pod \"84ebf521-8154-447b-b367-443f7dff3dfa\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " Jul 03 10:43:34.976077 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:34.975987 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-tmp-dir\") pod \"84ebf521-8154-447b-b367-443f7dff3dfa\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " Jul 03 10:43:34.976077 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:34.976033 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-dshm\") pod \"84ebf521-8154-447b-b367-443f7dff3dfa\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " Jul 03 10:43:34.976077 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:34.976059 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4gwc\" (UniqueName: \"kubernetes.io/projected/84ebf521-8154-447b-b367-443f7dff3dfa-kube-api-access-v4gwc\") pod \"84ebf521-8154-447b-b367-443f7dff3dfa\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " Jul 03 10:43:34.976291 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:34.976129 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84ebf521-8154-447b-b367-443f7dff3dfa-tls-certs\") pod \"84ebf521-8154-447b-b367-443f7dff3dfa\" (UID: \"84ebf521-8154-447b-b367-443f7dff3dfa\") " Jul 03 10:43:34.976291 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:34.976205 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "84ebf521-8154-447b-b367-443f7dff3dfa" (UID: "84ebf521-8154-447b-b367-443f7dff3dfa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:43:34.976399 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:34.976347 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-home" (OuterVolumeSpecName: "home") pod "84ebf521-8154-447b-b367-443f7dff3dfa" (UID: "84ebf521-8154-447b-b367-443f7dff3dfa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:43:34.976399 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:34.976363 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-model-cache" (OuterVolumeSpecName: "model-cache") pod "84ebf521-8154-447b-b367-443f7dff3dfa" (UID: "84ebf521-8154-447b-b367-443f7dff3dfa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:43:34.976492 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:34.976475 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-model-cache\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:43:34.976531 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:34.976494 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-home\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:43:34.976531 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:34.976504 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-tmp-dir\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:43:34.978309 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:34.978281 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84ebf521-8154-447b-b367-443f7dff3dfa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "84ebf521-8154-447b-b367-443f7dff3dfa" (UID: "84ebf521-8154-447b-b367-443f7dff3dfa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:43:34.978309 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:34.978281 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84ebf521-8154-447b-b367-443f7dff3dfa-kube-api-access-v4gwc" (OuterVolumeSpecName: "kube-api-access-v4gwc") pod "84ebf521-8154-447b-b367-443f7dff3dfa" (UID: "84ebf521-8154-447b-b367-443f7dff3dfa"). InnerVolumeSpecName "kube-api-access-v4gwc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:43:34.978447 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:34.978282 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-dshm" (OuterVolumeSpecName: "dshm") pod "84ebf521-8154-447b-b367-443f7dff3dfa" (UID: "84ebf521-8154-447b-b367-443f7dff3dfa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:43:35.077126 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:35.077027 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84ebf521-8154-447b-b367-443f7dff3dfa-dshm\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:43:35.077126 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:35.077059 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4gwc\" (UniqueName: \"kubernetes.io/projected/84ebf521-8154-447b-b367-443f7dff3dfa-kube-api-access-v4gwc\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:43:35.077126 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:35.077073 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84ebf521-8154-447b-b367-443f7dff3dfa-tls-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:43:35.762502 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:35.762476 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" Jul 03 10:43:35.762502 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:35.762497 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49" event={"ID":"84ebf521-8154-447b-b367-443f7dff3dfa","Type":"ContainerDied","Data":"962b3f0400631ab01337d243f355c516905a0784940fe46f8afa68dcb27fbeed"} Jul 03 10:43:35.763049 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:35.762543 2573 scope.go:117] "RemoveContainer" containerID="326a5cfc4448a151c317423ec7506b6a220f9249e7924ce98d48047a58d33939" Jul 03 10:43:35.799743 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:35.799707 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49"] Jul 03 10:43:35.804329 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:35.804298 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-cda536/auth-test-model-cda536-kserve-5b57449ff9-7dz49"] Jul 03 10:43:36.563685 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:36.563652 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84ebf521-8154-447b-b367-443f7dff3dfa" path="/var/lib/kubelet/pods/84ebf521-8154-447b-b367-443f7dff3dfa/volumes" Jul 03 10:43:39.004958 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:39.004925 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:39.004958 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:39.004967 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:39.017479 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:39.017453 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:43:39.794621 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:43:39.794581 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:44:38.576982 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:38.576886 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8"] Jul 03 10:44:53.596719 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:53.596662 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" podUID="48091010-a5fd-4ea9-bff4-91d1f7cb0618" containerName="main" containerID="cri-o://ee0dffd1e92194487a6856d83455edf2e69d3273581365752fc46fce3719822d" gracePeriod=45 Jul 03 10:44:53.637331 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:44:53.637294 2573 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48091010_a5fd_4ea9_bff4_91d1f7cb0618.slice/crio-ee0dffd1e92194487a6856d83455edf2e69d3273581365752fc46fce3719822d.scope\": RecentStats: unable to find data in memory cache]" Jul 03 10:44:53.849988 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:53.849922 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:44:53.945920 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:53.945882 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-dshm\") pod \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " Jul 03 10:44:53.946109 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:53.945973 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-tmp-dir\") pod \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " Jul 03 10:44:53.946109 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:53.945989 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-home\") pod \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " Jul 03 10:44:53.946109 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:53.946012 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qncx\" (UniqueName: \"kubernetes.io/projected/48091010-a5fd-4ea9-bff4-91d1f7cb0618-kube-api-access-9qncx\") pod \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " Jul 03 10:44:53.946109 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:53.946033 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-model-cache\") pod \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " Jul 03 10:44:53.946109 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:53.946085 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48091010-a5fd-4ea9-bff4-91d1f7cb0618-tls-certs\") pod \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\" (UID: \"48091010-a5fd-4ea9-bff4-91d1f7cb0618\") " Jul 03 10:44:53.946385 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:53.946297 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "48091010-a5fd-4ea9-bff4-91d1f7cb0618" (UID: "48091010-a5fd-4ea9-bff4-91d1f7cb0618"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:44:53.946385 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:53.946315 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-home" (OuterVolumeSpecName: "home") pod "48091010-a5fd-4ea9-bff4-91d1f7cb0618" (UID: "48091010-a5fd-4ea9-bff4-91d1f7cb0618"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:44:53.946385 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:53.946329 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-model-cache" (OuterVolumeSpecName: "model-cache") pod "48091010-a5fd-4ea9-bff4-91d1f7cb0618" (UID: "48091010-a5fd-4ea9-bff4-91d1f7cb0618"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:44:53.946548 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:53.946527 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-tmp-dir\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:44:53.946599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:53.946554 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-home\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:44:53.946599 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:53.946567 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-model-cache\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:44:53.948046 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:53.948017 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-dshm" (OuterVolumeSpecName: "dshm") pod "48091010-a5fd-4ea9-bff4-91d1f7cb0618" (UID: "48091010-a5fd-4ea9-bff4-91d1f7cb0618"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:44:53.948147 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:53.948069 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48091010-a5fd-4ea9-bff4-91d1f7cb0618-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "48091010-a5fd-4ea9-bff4-91d1f7cb0618" (UID: "48091010-a5fd-4ea9-bff4-91d1f7cb0618"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:44:53.948190 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:53.948172 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48091010-a5fd-4ea9-bff4-91d1f7cb0618-kube-api-access-9qncx" (OuterVolumeSpecName: "kube-api-access-9qncx") pod "48091010-a5fd-4ea9-bff4-91d1f7cb0618" (UID: "48091010-a5fd-4ea9-bff4-91d1f7cb0618"). InnerVolumeSpecName "kube-api-access-9qncx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:44:54.047832 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.047790 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48091010-a5fd-4ea9-bff4-91d1f7cb0618-dshm\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:44:54.047832 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.047824 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qncx\" (UniqueName: \"kubernetes.io/projected/48091010-a5fd-4ea9-bff4-91d1f7cb0618-kube-api-access-9qncx\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:44:54.047832 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.047842 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48091010-a5fd-4ea9-bff4-91d1f7cb0618-tls-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:44:54.126210 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.126131 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc"] Jul 03 10:44:54.126580 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.126567 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48091010-a5fd-4ea9-bff4-91d1f7cb0618" containerName="main" Jul 03 10:44:54.126626 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.126582 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="48091010-a5fd-4ea9-bff4-91d1f7cb0618" containerName="main" Jul 03 10:44:54.126626 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.126607 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84ebf521-8154-447b-b367-443f7dff3dfa" containerName="main" Jul 03 10:44:54.126626 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.126612 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="84ebf521-8154-447b-b367-443f7dff3dfa" containerName="main" Jul 03 10:44:54.126716 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.126672 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="48091010-a5fd-4ea9-bff4-91d1f7cb0618" containerName="main" Jul 03 10:44:54.126716 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.126681 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="84ebf521-8154-447b-b367-443f7dff3dfa" containerName="main" Jul 03 10:44:54.129760 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.129722 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:44:54.132942 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.132920 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-cda536\"/\"rate-test-model-cda536-kserve-self-signed-certs\"" Jul 03 10:44:54.144339 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.144316 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc"] Jul 03 10:44:54.166858 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.166822 2573 generic.go:356] "Generic (PLEG): container finished" podID="48091010-a5fd-4ea9-bff4-91d1f7cb0618" containerID="ee0dffd1e92194487a6856d83455edf2e69d3273581365752fc46fce3719822d" exitCode=0 Jul 03 10:44:54.167018 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.166882 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" Jul 03 10:44:54.167018 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.166906 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" event={"ID":"48091010-a5fd-4ea9-bff4-91d1f7cb0618","Type":"ContainerDied","Data":"ee0dffd1e92194487a6856d83455edf2e69d3273581365752fc46fce3719822d"} Jul 03 10:44:54.167018 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.166953 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8" event={"ID":"48091010-a5fd-4ea9-bff4-91d1f7cb0618","Type":"ContainerDied","Data":"bea9a0da16378159cbb07ee8cdba29b4d8394c29fb36a8763f3ab593df3ba47f"} Jul 03 10:44:54.167018 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.166975 2573 scope.go:117] "RemoveContainer" containerID="ee0dffd1e92194487a6856d83455edf2e69d3273581365752fc46fce3719822d" Jul 03 10:44:54.175594 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.175576 2573 scope.go:117] "RemoveContainer" containerID="ee0dffd1e92194487a6856d83455edf2e69d3273581365752fc46fce3719822d" Jul 03 10:44:54.175882 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:44:54.175855 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee0dffd1e92194487a6856d83455edf2e69d3273581365752fc46fce3719822d\": container with ID starting with ee0dffd1e92194487a6856d83455edf2e69d3273581365752fc46fce3719822d not found: ID does not exist" containerID="ee0dffd1e92194487a6856d83455edf2e69d3273581365752fc46fce3719822d" Jul 03 10:44:54.175951 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.175890 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee0dffd1e92194487a6856d83455edf2e69d3273581365752fc46fce3719822d"} err="failed to get container status \"ee0dffd1e92194487a6856d83455edf2e69d3273581365752fc46fce3719822d\": rpc error: code = NotFound desc = could not find container \"ee0dffd1e92194487a6856d83455edf2e69d3273581365752fc46fce3719822d\": container with ID starting with ee0dffd1e92194487a6856d83455edf2e69d3273581365752fc46fce3719822d not found: ID does not exist" Jul 03 10:44:54.203510 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.203476 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8"] Jul 03 10:44:54.209092 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.209065 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-cda536/sub-test-model-cda536-kserve-789b4c4587-9sxt8"] Jul 03 10:44:54.249704 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.249671 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-home\") pod \"rate-test-model-cda536-kserve-65c8566875-zk5qc\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:44:54.249908 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.249748 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-dshm\") pod \"rate-test-model-cda536-kserve-65c8566875-zk5qc\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:44:54.249908 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.249778 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2qc5\" (UniqueName: \"kubernetes.io/projected/0da2b8ef-5f9f-473f-b5e9-832f5564336b-kube-api-access-j2qc5\") pod \"rate-test-model-cda536-kserve-65c8566875-zk5qc\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:44:54.249908 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.249819 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-tmp-dir\") pod \"rate-test-model-cda536-kserve-65c8566875-zk5qc\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:44:54.249908 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.249885 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-model-cache\") pod \"rate-test-model-cda536-kserve-65c8566875-zk5qc\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:44:54.250068 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.249959 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0da2b8ef-5f9f-473f-b5e9-832f5564336b-tls-certs\") pod \"rate-test-model-cda536-kserve-65c8566875-zk5qc\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:44:54.351332 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.351290 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-home\") pod \"rate-test-model-cda536-kserve-65c8566875-zk5qc\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:44:54.351533 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.351355 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-dshm\") pod \"rate-test-model-cda536-kserve-65c8566875-zk5qc\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:44:54.351533 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.351387 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2qc5\" (UniqueName: \"kubernetes.io/projected/0da2b8ef-5f9f-473f-b5e9-832f5564336b-kube-api-access-j2qc5\") pod \"rate-test-model-cda536-kserve-65c8566875-zk5qc\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:44:54.351533 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.351426 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-tmp-dir\") pod \"rate-test-model-cda536-kserve-65c8566875-zk5qc\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:44:54.351533 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.351460 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-model-cache\") pod \"rate-test-model-cda536-kserve-65c8566875-zk5qc\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:44:54.351533 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.351532 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0da2b8ef-5f9f-473f-b5e9-832f5564336b-tls-certs\") pod \"rate-test-model-cda536-kserve-65c8566875-zk5qc\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:44:54.351850 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.351704 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-home\") pod \"rate-test-model-cda536-kserve-65c8566875-zk5qc\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:44:54.351915 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.351867 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-tmp-dir\") pod \"rate-test-model-cda536-kserve-65c8566875-zk5qc\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:44:54.351915 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.351908 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-model-cache\") pod \"rate-test-model-cda536-kserve-65c8566875-zk5qc\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:44:54.353544 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.353514 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-dshm\") pod \"rate-test-model-cda536-kserve-65c8566875-zk5qc\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:44:54.353940 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.353917 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0da2b8ef-5f9f-473f-b5e9-832f5564336b-tls-certs\") pod \"rate-test-model-cda536-kserve-65c8566875-zk5qc\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:44:54.359871 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.359853 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2qc5\" (UniqueName: \"kubernetes.io/projected/0da2b8ef-5f9f-473f-b5e9-832f5564336b-kube-api-access-j2qc5\") pod \"rate-test-model-cda536-kserve-65c8566875-zk5qc\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:44:54.438672 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.438636 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:44:54.564171 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.564138 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48091010-a5fd-4ea9-bff4-91d1f7cb0618" path="/var/lib/kubelet/pods/48091010-a5fd-4ea9-bff4-91d1f7cb0618/volumes" Jul 03 10:44:54.576450 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:54.576426 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc"] Jul 03 10:44:54.579152 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:44:54.579128 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0da2b8ef_5f9f_473f_b5e9_832f5564336b.slice/crio-52fea01c819729f8769c05386a7fa7b5379aa075e804548544a77a2762d775e9 WatchSource:0}: Error finding container 52fea01c819729f8769c05386a7fa7b5379aa075e804548544a77a2762d775e9: Status 404 returned error can't find the container with id 52fea01c819729f8769c05386a7fa7b5379aa075e804548544a77a2762d775e9 Jul 03 10:44:55.173170 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:55.173129 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" event={"ID":"0da2b8ef-5f9f-473f-b5e9-832f5564336b","Type":"ContainerStarted","Data":"a5472f9cca1ec6009b26c57eac3d94ad2ff663cd67d87bdae36c401a862d66dd"} Jul 03 10:44:55.173645 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:55.173180 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" event={"ID":"0da2b8ef-5f9f-473f-b5e9-832f5564336b","Type":"ContainerStarted","Data":"52fea01c819729f8769c05386a7fa7b5379aa075e804548544a77a2762d775e9"} Jul 03 10:44:55.193591 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:44:55.193478 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" podStartSLOduration=1.19346372 podStartE2EDuration="1.19346372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:44:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:44:55.193420322 +0000 UTC m=+2350.105309628" watchObservedRunningTime="2026-07-03 10:44:55.19346372 +0000 UTC m=+2350.105353024" Jul 03 10:45:00.133804 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.133768 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-cda536-29717925-6jrpj"] Jul 03 10:45:00.137413 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.137390 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-cda536-29717925-6jrpj" Jul 03 10:45:00.139029 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.139006 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29717925-d6276"] Jul 03 10:45:00.142809 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.142767 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-62c45e-29717925-tbvww"] Jul 03 10:45:00.142925 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.142880 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717925-d6276" Jul 03 10:45:00.146885 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.146864 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-cda536-29717925-6jrpj"] Jul 03 10:45:00.146975 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.146955 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-62c45e-29717925-tbvww" Jul 03 10:45:00.151951 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.151923 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-62c45e-29717925-tbvww"] Jul 03 10:45:00.160522 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.160495 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717925-d6276"] Jul 03 10:45:00.206860 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.206822 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqvx8\" (UniqueName: \"kubernetes.io/projected/8b46b71c-add7-4029-a328-98149707b8f2-kube-api-access-hqvx8\") pod \"maas-api-key-cleanup-29717925-d6276\" (UID: \"8b46b71c-add7-4029-a328-98149707b8f2\") " pod="opendatahub/maas-api-key-cleanup-29717925-d6276" Jul 03 10:45:00.207049 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.206903 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62h68\" (UniqueName: \"kubernetes.io/projected/62e2d2e5-6f7a-49a8-a32a-7227e8cf108b-kube-api-access-62h68\") pod \"maas-api-key-cleanup-e2e-shared-a-cda536-29717925-6jrpj\" (UID: \"62e2d2e5-6f7a-49a8-a32a-7227e8cf108b\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-cda536-29717925-6jrpj" Jul 03 10:45:00.207049 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.206945 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkdtz\" (UniqueName: \"kubernetes.io/projected/194e39df-559b-4320-a4d5-0cc0a3909ac3-kube-api-access-dkdtz\") pod \"maas-api-key-cleanup-e2e-shared-b-62c45e-29717925-tbvww\" (UID: \"194e39df-559b-4320-a4d5-0cc0a3909ac3\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-62c45e-29717925-tbvww" Jul 03 10:45:00.308282 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.308248 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqvx8\" (UniqueName: \"kubernetes.io/projected/8b46b71c-add7-4029-a328-98149707b8f2-kube-api-access-hqvx8\") pod \"maas-api-key-cleanup-29717925-d6276\" (UID: \"8b46b71c-add7-4029-a328-98149707b8f2\") " pod="opendatahub/maas-api-key-cleanup-29717925-d6276" Jul 03 10:45:00.308487 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.308319 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62h68\" (UniqueName: \"kubernetes.io/projected/62e2d2e5-6f7a-49a8-a32a-7227e8cf108b-kube-api-access-62h68\") pod \"maas-api-key-cleanup-e2e-shared-a-cda536-29717925-6jrpj\" (UID: \"62e2d2e5-6f7a-49a8-a32a-7227e8cf108b\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-cda536-29717925-6jrpj" Jul 03 10:45:00.308487 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.308356 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkdtz\" (UniqueName: \"kubernetes.io/projected/194e39df-559b-4320-a4d5-0cc0a3909ac3-kube-api-access-dkdtz\") pod \"maas-api-key-cleanup-e2e-shared-b-62c45e-29717925-tbvww\" (UID: \"194e39df-559b-4320-a4d5-0cc0a3909ac3\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-62c45e-29717925-tbvww" Jul 03 10:45:00.318097 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.318064 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkdtz\" (UniqueName: \"kubernetes.io/projected/194e39df-559b-4320-a4d5-0cc0a3909ac3-kube-api-access-dkdtz\") pod \"maas-api-key-cleanup-e2e-shared-b-62c45e-29717925-tbvww\" (UID: \"194e39df-559b-4320-a4d5-0cc0a3909ac3\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-62c45e-29717925-tbvww" Jul 03 10:45:00.318223 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.318065 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62h68\" (UniqueName: \"kubernetes.io/projected/62e2d2e5-6f7a-49a8-a32a-7227e8cf108b-kube-api-access-62h68\") pod \"maas-api-key-cleanup-e2e-shared-a-cda536-29717925-6jrpj\" (UID: \"62e2d2e5-6f7a-49a8-a32a-7227e8cf108b\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-cda536-29717925-6jrpj" Jul 03 10:45:00.318223 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.318145 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqvx8\" (UniqueName: \"kubernetes.io/projected/8b46b71c-add7-4029-a328-98149707b8f2-kube-api-access-hqvx8\") pod \"maas-api-key-cleanup-29717925-d6276\" (UID: \"8b46b71c-add7-4029-a328-98149707b8f2\") " pod="opendatahub/maas-api-key-cleanup-29717925-d6276" Jul 03 10:45:00.452335 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.452244 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-cda536-29717925-6jrpj" Jul 03 10:45:00.463360 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.463336 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717925-d6276" Jul 03 10:45:00.471579 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.471552 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-62c45e-29717925-tbvww" Jul 03 10:45:00.613046 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.613021 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-cda536-29717925-6jrpj"] Jul 03 10:45:00.615202 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:45:00.615178 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62e2d2e5_6f7a_49a8_a32a_7227e8cf108b.slice/crio-bfda93ab008324b6a4644fafda16baf2003d2c3f05a5f74cd2ca78bc30383b8d WatchSource:0}: Error finding container bfda93ab008324b6a4644fafda16baf2003d2c3f05a5f74cd2ca78bc30383b8d: Status 404 returned error can't find the container with id bfda93ab008324b6a4644fafda16baf2003d2c3f05a5f74cd2ca78bc30383b8d Jul 03 10:45:00.839992 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.839952 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717925-d6276"] Jul 03 10:45:00.842017 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:45:00.841987 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b46b71c_add7_4029_a328_98149707b8f2.slice/crio-0de4bdb14f213eeaa1c7276c1f8cb94395554ff1c5eb39b7760f92a489dd1b52 WatchSource:0}: Error finding container 0de4bdb14f213eeaa1c7276c1f8cb94395554ff1c5eb39b7760f92a489dd1b52: Status 404 returned error can't find the container with id 0de4bdb14f213eeaa1c7276c1f8cb94395554ff1c5eb39b7760f92a489dd1b52 Jul 03 10:45:00.842894 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:00.842865 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-62c45e-29717925-tbvww"] Jul 03 10:45:00.844283 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:45:00.844256 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod194e39df_559b_4320_a4d5_0cc0a3909ac3.slice/crio-aadc9dbb0cbcf711e5923fbb6aa2fedf8df9ca2a5b41e8f263757849d7597ea1 WatchSource:0}: Error finding container aadc9dbb0cbcf711e5923fbb6aa2fedf8df9ca2a5b41e8f263757849d7597ea1: Status 404 returned error can't find the container with id aadc9dbb0cbcf711e5923fbb6aa2fedf8df9ca2a5b41e8f263757849d7597ea1 Jul 03 10:45:01.206535 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:01.206483 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-cda536-29717925-6jrpj" event={"ID":"62e2d2e5-6f7a-49a8-a32a-7227e8cf108b","Type":"ContainerStarted","Data":"27c6832658bc85e31bb38be4d4a5706594398f760d939eeb1867fa9956ca23ae"} Jul 03 10:45:01.207018 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:01.206584 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-cda536-29717925-6jrpj" event={"ID":"62e2d2e5-6f7a-49a8-a32a-7227e8cf108b","Type":"ContainerStarted","Data":"bfda93ab008324b6a4644fafda16baf2003d2c3f05a5f74cd2ca78bc30383b8d"} Jul 03 10:45:01.208051 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:01.208015 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717925-d6276" event={"ID":"8b46b71c-add7-4029-a328-98149707b8f2","Type":"ContainerStarted","Data":"55814da21182f9292cae427f32459d898987abeed10558e4bfb44a33ae756a84"} Jul 03 10:45:01.208189 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:01.208059 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717925-d6276" event={"ID":"8b46b71c-add7-4029-a328-98149707b8f2","Type":"ContainerStarted","Data":"0de4bdb14f213eeaa1c7276c1f8cb94395554ff1c5eb39b7760f92a489dd1b52"} Jul 03 10:45:01.209145 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:01.209124 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-62c45e-29717925-tbvww" event={"ID":"194e39df-559b-4320-a4d5-0cc0a3909ac3","Type":"ContainerStarted","Data":"aadc9dbb0cbcf711e5923fbb6aa2fedf8df9ca2a5b41e8f263757849d7597ea1"} Jul 03 10:45:01.223147 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:01.223098 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-cda536-29717925-6jrpj" podStartSLOduration=1.223082769 podStartE2EDuration="1.223082769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:45:01.222285711 +0000 UTC m=+2356.134175015" watchObservedRunningTime="2026-07-03 10:45:01.223082769 +0000 UTC m=+2356.134972074" Jul 03 10:45:01.241098 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:01.241056 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29717925-d6276" podStartSLOduration=1.241044087 podStartE2EDuration="1.241044087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:45:01.23890976 +0000 UTC m=+2356.150799066" watchObservedRunningTime="2026-07-03 10:45:01.241044087 +0000 UTC m=+2356.152933390" Jul 03 10:45:02.216788 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:02.216754 2573 generic.go:356] "Generic (PLEG): container finished" podID="194e39df-559b-4320-a4d5-0cc0a3909ac3" containerID="811dfd99aa206f46aa9c693be1ffca922a9a48db5756825271251cec533365f0" exitCode=0 Jul 03 10:45:02.217236 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:02.216825 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-62c45e-29717925-tbvww" event={"ID":"194e39df-559b-4320-a4d5-0cc0a3909ac3","Type":"ContainerDied","Data":"811dfd99aa206f46aa9c693be1ffca922a9a48db5756825271251cec533365f0"} Jul 03 10:45:02.218328 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:02.218304 2573 generic.go:356] "Generic (PLEG): container finished" podID="62e2d2e5-6f7a-49a8-a32a-7227e8cf108b" containerID="27c6832658bc85e31bb38be4d4a5706594398f760d939eeb1867fa9956ca23ae" exitCode=0 Jul 03 10:45:02.218456 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:02.218387 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-cda536-29717925-6jrpj" event={"ID":"62e2d2e5-6f7a-49a8-a32a-7227e8cf108b","Type":"ContainerDied","Data":"27c6832658bc85e31bb38be4d4a5706594398f760d939eeb1867fa9956ca23ae"} Jul 03 10:45:02.219723 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:02.219704 2573 generic.go:356] "Generic (PLEG): container finished" podID="8b46b71c-add7-4029-a328-98149707b8f2" containerID="55814da21182f9292cae427f32459d898987abeed10558e4bfb44a33ae756a84" exitCode=0 Jul 03 10:45:02.219833 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:02.219763 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717925-d6276" event={"ID":"8b46b71c-add7-4029-a328-98149707b8f2","Type":"ContainerDied","Data":"55814da21182f9292cae427f32459d898987abeed10558e4bfb44a33ae756a84"} Jul 03 10:45:03.380025 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:03.379985 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-62c45e-29717925-tbvww" Jul 03 10:45:03.411449 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:03.411422 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717925-d6276" Jul 03 10:45:03.414756 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:03.414715 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-cda536-29717925-6jrpj" Jul 03 10:45:03.437069 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:03.437040 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqvx8\" (UniqueName: \"kubernetes.io/projected/8b46b71c-add7-4029-a328-98149707b8f2-kube-api-access-hqvx8\") pod \"8b46b71c-add7-4029-a328-98149707b8f2\" (UID: \"8b46b71c-add7-4029-a328-98149707b8f2\") " Jul 03 10:45:03.437236 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:03.437115 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkdtz\" (UniqueName: \"kubernetes.io/projected/194e39df-559b-4320-a4d5-0cc0a3909ac3-kube-api-access-dkdtz\") pod \"194e39df-559b-4320-a4d5-0cc0a3909ac3\" (UID: \"194e39df-559b-4320-a4d5-0cc0a3909ac3\") " Jul 03 10:45:03.437236 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:03.437202 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62h68\" (UniqueName: \"kubernetes.io/projected/62e2d2e5-6f7a-49a8-a32a-7227e8cf108b-kube-api-access-62h68\") pod \"62e2d2e5-6f7a-49a8-a32a-7227e8cf108b\" (UID: \"62e2d2e5-6f7a-49a8-a32a-7227e8cf108b\") " Jul 03 10:45:03.439172 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:03.439142 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62e2d2e5-6f7a-49a8-a32a-7227e8cf108b-kube-api-access-62h68" (OuterVolumeSpecName: "kube-api-access-62h68") pod "62e2d2e5-6f7a-49a8-a32a-7227e8cf108b" (UID: "62e2d2e5-6f7a-49a8-a32a-7227e8cf108b"). InnerVolumeSpecName "kube-api-access-62h68". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:45:03.439271 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:03.439156 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b46b71c-add7-4029-a328-98149707b8f2-kube-api-access-hqvx8" (OuterVolumeSpecName: "kube-api-access-hqvx8") pod "8b46b71c-add7-4029-a328-98149707b8f2" (UID: "8b46b71c-add7-4029-a328-98149707b8f2"). InnerVolumeSpecName "kube-api-access-hqvx8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:45:03.439271 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:03.439171 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/194e39df-559b-4320-a4d5-0cc0a3909ac3-kube-api-access-dkdtz" (OuterVolumeSpecName: "kube-api-access-dkdtz") pod "194e39df-559b-4320-a4d5-0cc0a3909ac3" (UID: "194e39df-559b-4320-a4d5-0cc0a3909ac3"). InnerVolumeSpecName "kube-api-access-dkdtz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:45:03.538796 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:03.538676 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkdtz\" (UniqueName: \"kubernetes.io/projected/194e39df-559b-4320-a4d5-0cc0a3909ac3-kube-api-access-dkdtz\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:45:03.538796 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:03.538706 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62h68\" (UniqueName: \"kubernetes.io/projected/62e2d2e5-6f7a-49a8-a32a-7227e8cf108b-kube-api-access-62h68\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:45:03.538796 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:03.538715 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqvx8\" (UniqueName: \"kubernetes.io/projected/8b46b71c-add7-4029-a328-98149707b8f2-kube-api-access-hqvx8\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:45:04.232076 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:04.232045 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717925-d6276" Jul 03 10:45:04.232225 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:04.232078 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717925-d6276" event={"ID":"8b46b71c-add7-4029-a328-98149707b8f2","Type":"ContainerDied","Data":"0de4bdb14f213eeaa1c7276c1f8cb94395554ff1c5eb39b7760f92a489dd1b52"} Jul 03 10:45:04.232225 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:04.232139 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0de4bdb14f213eeaa1c7276c1f8cb94395554ff1c5eb39b7760f92a489dd1b52" Jul 03 10:45:04.233529 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:04.233511 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-62c45e-29717925-tbvww" Jul 03 10:45:04.233666 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:04.233543 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-62c45e-29717925-tbvww" event={"ID":"194e39df-559b-4320-a4d5-0cc0a3909ac3","Type":"ContainerDied","Data":"aadc9dbb0cbcf711e5923fbb6aa2fedf8df9ca2a5b41e8f263757849d7597ea1"} Jul 03 10:45:04.233666 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:04.233586 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aadc9dbb0cbcf711e5923fbb6aa2fedf8df9ca2a5b41e8f263757849d7597ea1" Jul 03 10:45:04.234957 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:04.234933 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-cda536-29717925-6jrpj" event={"ID":"62e2d2e5-6f7a-49a8-a32a-7227e8cf108b","Type":"ContainerDied","Data":"bfda93ab008324b6a4644fafda16baf2003d2c3f05a5f74cd2ca78bc30383b8d"} Jul 03 10:45:04.234957 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:04.234953 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-cda536-29717925-6jrpj" Jul 03 10:45:04.234957 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:04.234970 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bfda93ab008324b6a4644fafda16baf2003d2c3f05a5f74cd2ca78bc30383b8d" Jul 03 10:45:04.438918 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:04.438884 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:45:04.439289 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:04.438934 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:45:04.451399 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:04.451375 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:45:05.251603 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:05.251577 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:45:55.560763 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:55.560715 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc"] Jul 03 10:45:58.219592 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:58.219556 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd"] Jul 03 10:45:58.220100 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:58.219857 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" podUID="569893b6-194a-41e3-8bd0-9563864ea7ea" containerName="maas-api" containerID="cri-o://8b8a173fe80b29d80283f0606cb03f3ab4b9c7d58b40e8c044e2a00d5c3810bf" gracePeriod=30 Jul 03 10:45:58.247802 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:58.247765 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-cda536-29717925-6jrpj"] Jul 03 10:45:58.250285 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:58.250259 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-cda536-29717925-6jrpj"] Jul 03 10:45:58.565174 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:58.565137 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62e2d2e5-6f7a-49a8-a32a-7227e8cf108b" path="/var/lib/kubelet/pods/62e2d2e5-6f7a-49a8-a32a-7227e8cf108b/volumes" Jul 03 10:45:58.671143 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:58.671120 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" Jul 03 10:45:58.780269 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:58.780177 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/569893b6-194a-41e3-8bd0-9563864ea7ea-openshift-service-ca\") pod \"569893b6-194a-41e3-8bd0-9563864ea7ea\" (UID: \"569893b6-194a-41e3-8bd0-9563864ea7ea\") " Jul 03 10:45:58.780269 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:58.780215 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/569893b6-194a-41e3-8bd0-9563864ea7ea-maas-api-tls\") pod \"569893b6-194a-41e3-8bd0-9563864ea7ea\" (UID: \"569893b6-194a-41e3-8bd0-9563864ea7ea\") " Jul 03 10:45:58.780470 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:58.780279 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49klz\" (UniqueName: \"kubernetes.io/projected/569893b6-194a-41e3-8bd0-9563864ea7ea-kube-api-access-49klz\") pod \"569893b6-194a-41e3-8bd0-9563864ea7ea\" (UID: \"569893b6-194a-41e3-8bd0-9563864ea7ea\") " Jul 03 10:45:58.780559 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:58.780537 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/569893b6-194a-41e3-8bd0-9563864ea7ea-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "569893b6-194a-41e3-8bd0-9563864ea7ea" (UID: "569893b6-194a-41e3-8bd0-9563864ea7ea"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:45:58.782485 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:58.782462 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/569893b6-194a-41e3-8bd0-9563864ea7ea-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "569893b6-194a-41e3-8bd0-9563864ea7ea" (UID: "569893b6-194a-41e3-8bd0-9563864ea7ea"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:45:58.782485 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:58.782466 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/569893b6-194a-41e3-8bd0-9563864ea7ea-kube-api-access-49klz" (OuterVolumeSpecName: "kube-api-access-49klz") pod "569893b6-194a-41e3-8bd0-9563864ea7ea" (UID: "569893b6-194a-41e3-8bd0-9563864ea7ea"). InnerVolumeSpecName "kube-api-access-49klz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:45:58.881630 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:58.881590 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/569893b6-194a-41e3-8bd0-9563864ea7ea-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:45:58.881630 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:58.881622 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/569893b6-194a-41e3-8bd0-9563864ea7ea-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:45:58.881630 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:58.881637 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49klz\" (UniqueName: \"kubernetes.io/projected/569893b6-194a-41e3-8bd0-9563864ea7ea-kube-api-access-49klz\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:45:59.527389 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:59.527349 2573 generic.go:356] "Generic (PLEG): container finished" podID="569893b6-194a-41e3-8bd0-9563864ea7ea" containerID="8b8a173fe80b29d80283f0606cb03f3ab4b9c7d58b40e8c044e2a00d5c3810bf" exitCode=0 Jul 03 10:45:59.527844 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:59.527396 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" Jul 03 10:45:59.527844 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:59.527399 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" event={"ID":"569893b6-194a-41e3-8bd0-9563864ea7ea","Type":"ContainerDied","Data":"8b8a173fe80b29d80283f0606cb03f3ab4b9c7d58b40e8c044e2a00d5c3810bf"} Jul 03 10:45:59.527844 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:59.527451 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd" event={"ID":"569893b6-194a-41e3-8bd0-9563864ea7ea","Type":"ContainerDied","Data":"00ab1c9d40747756f2bc1fa3150423542eddd1c4ba07affa47e8e9d11d4abee7"} Jul 03 10:45:59.527844 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:59.527481 2573 scope.go:117] "RemoveContainer" containerID="8b8a173fe80b29d80283f0606cb03f3ab4b9c7d58b40e8c044e2a00d5c3810bf" Jul 03 10:45:59.537058 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:59.537039 2573 scope.go:117] "RemoveContainer" containerID="8b8a173fe80b29d80283f0606cb03f3ab4b9c7d58b40e8c044e2a00d5c3810bf" Jul 03 10:45:59.537315 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:45:59.537291 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b8a173fe80b29d80283f0606cb03f3ab4b9c7d58b40e8c044e2a00d5c3810bf\": container with ID starting with 8b8a173fe80b29d80283f0606cb03f3ab4b9c7d58b40e8c044e2a00d5c3810bf not found: ID does not exist" containerID="8b8a173fe80b29d80283f0606cb03f3ab4b9c7d58b40e8c044e2a00d5c3810bf" Jul 03 10:45:59.537360 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:59.537326 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b8a173fe80b29d80283f0606cb03f3ab4b9c7d58b40e8c044e2a00d5c3810bf"} err="failed to get container status \"8b8a173fe80b29d80283f0606cb03f3ab4b9c7d58b40e8c044e2a00d5c3810bf\": rpc error: code = NotFound desc = could not find container \"8b8a173fe80b29d80283f0606cb03f3ab4b9c7d58b40e8c044e2a00d5c3810bf\": container with ID starting with 8b8a173fe80b29d80283f0606cb03f3ab4b9c7d58b40e8c044e2a00d5c3810bf not found: ID does not exist" Jul 03 10:45:59.549957 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:59.549931 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd"] Jul 03 10:45:59.553702 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:45:59.553679 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-cda536-574b4fd5fc-wgvhd"] Jul 03 10:46:00.563518 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:00.563485 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="569893b6-194a-41e3-8bd0-9563864ea7ea" path="/var/lib/kubelet/pods/569893b6-194a-41e3-8bd0-9563864ea7ea/volumes" Jul 03 10:46:10.581619 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:10.581515 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" podUID="0da2b8ef-5f9f-473f-b5e9-832f5564336b" containerName="main" containerID="cri-o://a5472f9cca1ec6009b26c57eac3d94ad2ff663cd67d87bdae36c401a862d66dd" gracePeriod=45 Jul 03 10:46:10.831542 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:10.831514 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:46:10.997519 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:10.997484 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0da2b8ef-5f9f-473f-b5e9-832f5564336b-tls-certs\") pod \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " Jul 03 10:46:10.997775 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:10.997550 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-home\") pod \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " Jul 03 10:46:10.997775 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:10.997571 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-dshm\") pod \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " Jul 03 10:46:10.997775 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:10.997594 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2qc5\" (UniqueName: \"kubernetes.io/projected/0da2b8ef-5f9f-473f-b5e9-832f5564336b-kube-api-access-j2qc5\") pod \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " Jul 03 10:46:10.997775 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:10.997622 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-tmp-dir\") pod \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " Jul 03 10:46:10.997775 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:10.997675 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-model-cache\") pod \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\" (UID: \"0da2b8ef-5f9f-473f-b5e9-832f5564336b\") " Jul 03 10:46:10.998065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:10.997906 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-home" (OuterVolumeSpecName: "home") pod "0da2b8ef-5f9f-473f-b5e9-832f5564336b" (UID: "0da2b8ef-5f9f-473f-b5e9-832f5564336b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:46:10.998065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:10.997974 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0da2b8ef-5f9f-473f-b5e9-832f5564336b" (UID: "0da2b8ef-5f9f-473f-b5e9-832f5564336b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:46:10.998065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:10.998028 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-model-cache" (OuterVolumeSpecName: "model-cache") pod "0da2b8ef-5f9f-473f-b5e9-832f5564336b" (UID: "0da2b8ef-5f9f-473f-b5e9-832f5564336b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:46:10.998214 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:10.998072 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-home\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:46:10.999853 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:10.999832 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-dshm" (OuterVolumeSpecName: "dshm") pod "0da2b8ef-5f9f-473f-b5e9-832f5564336b" (UID: "0da2b8ef-5f9f-473f-b5e9-832f5564336b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 03 10:46:10.999853 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:10.999830 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0da2b8ef-5f9f-473f-b5e9-832f5564336b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0da2b8ef-5f9f-473f-b5e9-832f5564336b" (UID: "0da2b8ef-5f9f-473f-b5e9-832f5564336b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:46:10.999988 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:10.999855 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0da2b8ef-5f9f-473f-b5e9-832f5564336b-kube-api-access-j2qc5" (OuterVolumeSpecName: "kube-api-access-j2qc5") pod "0da2b8ef-5f9f-473f-b5e9-832f5564336b" (UID: "0da2b8ef-5f9f-473f-b5e9-832f5564336b"). InnerVolumeSpecName "kube-api-access-j2qc5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:46:11.099153 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:11.099114 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-dshm\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:46:11.099153 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:11.099148 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2qc5\" (UniqueName: \"kubernetes.io/projected/0da2b8ef-5f9f-473f-b5e9-832f5564336b-kube-api-access-j2qc5\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:46:11.099372 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:11.099163 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-tmp-dir\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:46:11.099372 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:11.099176 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0da2b8ef-5f9f-473f-b5e9-832f5564336b-model-cache\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:46:11.099372 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:11.099188 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0da2b8ef-5f9f-473f-b5e9-832f5564336b-tls-certs\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:46:11.595065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:11.595029 2573 generic.go:356] "Generic (PLEG): container finished" podID="0da2b8ef-5f9f-473f-b5e9-832f5564336b" containerID="a5472f9cca1ec6009b26c57eac3d94ad2ff663cd67d87bdae36c401a862d66dd" exitCode=0 Jul 03 10:46:11.595457 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:11.595071 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" Jul 03 10:46:11.595457 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:11.595082 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" event={"ID":"0da2b8ef-5f9f-473f-b5e9-832f5564336b","Type":"ContainerDied","Data":"a5472f9cca1ec6009b26c57eac3d94ad2ff663cd67d87bdae36c401a862d66dd"} Jul 03 10:46:11.595457 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:11.595121 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc" event={"ID":"0da2b8ef-5f9f-473f-b5e9-832f5564336b","Type":"ContainerDied","Data":"52fea01c819729f8769c05386a7fa7b5379aa075e804548544a77a2762d775e9"} Jul 03 10:46:11.595457 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:11.595139 2573 scope.go:117] "RemoveContainer" containerID="a5472f9cca1ec6009b26c57eac3d94ad2ff663cd67d87bdae36c401a862d66dd" Jul 03 10:46:11.604497 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:11.604481 2573 scope.go:117] "RemoveContainer" containerID="a5472f9cca1ec6009b26c57eac3d94ad2ff663cd67d87bdae36c401a862d66dd" Jul 03 10:46:11.604796 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:46:11.604777 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5472f9cca1ec6009b26c57eac3d94ad2ff663cd67d87bdae36c401a862d66dd\": container with ID starting with a5472f9cca1ec6009b26c57eac3d94ad2ff663cd67d87bdae36c401a862d66dd not found: ID does not exist" containerID="a5472f9cca1ec6009b26c57eac3d94ad2ff663cd67d87bdae36c401a862d66dd" Jul 03 10:46:11.604852 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:11.604805 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5472f9cca1ec6009b26c57eac3d94ad2ff663cd67d87bdae36c401a862d66dd"} err="failed to get container status \"a5472f9cca1ec6009b26c57eac3d94ad2ff663cd67d87bdae36c401a862d66dd\": rpc error: code = NotFound desc = could not find container \"a5472f9cca1ec6009b26c57eac3d94ad2ff663cd67d87bdae36c401a862d66dd\": container with ID starting with a5472f9cca1ec6009b26c57eac3d94ad2ff663cd67d87bdae36c401a862d66dd not found: ID does not exist" Jul 03 10:46:11.619282 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:11.619253 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc"] Jul 03 10:46:11.623746 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:11.623709 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-cda536/rate-test-model-cda536-kserve-65c8566875-zk5qc"] Jul 03 10:46:12.568863 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:12.568833 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0da2b8ef-5f9f-473f-b5e9-832f5564336b" path="/var/lib/kubelet/pods/0da2b8ef-5f9f-473f-b5e9-832f5564336b/volumes" Jul 03 10:46:41.964357 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:41.964318 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk"] Jul 03 10:46:41.964935 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:41.964616 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" podUID="c9565699-e375-4d15-b61d-82d015b96298" containerName="maas-api" containerID="cri-o://5305a66390d8facbd26ce094250a124446b1161bedba03fe66a03e5b2bdd781e" gracePeriod=30 Jul 03 10:46:41.999340 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:41.999302 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-62c45e-29717925-tbvww"] Jul 03 10:46:42.001036 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.001012 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-62c45e-29717925-tbvww"] Jul 03 10:46:42.218403 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.218333 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" Jul 03 10:46:42.309187 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.309152 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9565699-e375-4d15-b61d-82d015b96298-openshift-service-ca\") pod \"c9565699-e375-4d15-b61d-82d015b96298\" (UID: \"c9565699-e375-4d15-b61d-82d015b96298\") " Jul 03 10:46:42.309364 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.309197 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t52qc\" (UniqueName: \"kubernetes.io/projected/c9565699-e375-4d15-b61d-82d015b96298-kube-api-access-t52qc\") pod \"c9565699-e375-4d15-b61d-82d015b96298\" (UID: \"c9565699-e375-4d15-b61d-82d015b96298\") " Jul 03 10:46:42.309364 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.309271 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9565699-e375-4d15-b61d-82d015b96298-maas-api-tls\") pod \"c9565699-e375-4d15-b61d-82d015b96298\" (UID: \"c9565699-e375-4d15-b61d-82d015b96298\") " Jul 03 10:46:42.309605 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.309581 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9565699-e375-4d15-b61d-82d015b96298-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c9565699-e375-4d15-b61d-82d015b96298" (UID: "c9565699-e375-4d15-b61d-82d015b96298"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 03 10:46:42.311299 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.311268 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9565699-e375-4d15-b61d-82d015b96298-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c9565699-e375-4d15-b61d-82d015b96298" (UID: "c9565699-e375-4d15-b61d-82d015b96298"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 03 10:46:42.311411 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.311308 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9565699-e375-4d15-b61d-82d015b96298-kube-api-access-t52qc" (OuterVolumeSpecName: "kube-api-access-t52qc") pod "c9565699-e375-4d15-b61d-82d015b96298" (UID: "c9565699-e375-4d15-b61d-82d015b96298"). InnerVolumeSpecName "kube-api-access-t52qc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 03 10:46:42.410875 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.410839 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t52qc\" (UniqueName: \"kubernetes.io/projected/c9565699-e375-4d15-b61d-82d015b96298-kube-api-access-t52qc\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:46:42.410875 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.410870 2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9565699-e375-4d15-b61d-82d015b96298-maas-api-tls\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:46:42.410875 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.410880 2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9565699-e375-4d15-b61d-82d015b96298-openshift-service-ca\") on node \"ip-10-0-134-191.ec2.internal\" DevicePath \"\"" Jul 03 10:46:42.563810 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.563701 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="194e39df-559b-4320-a4d5-0cc0a3909ac3" path="/var/lib/kubelet/pods/194e39df-559b-4320-a4d5-0cc0a3909ac3/volumes" Jul 03 10:46:42.755629 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.755589 2573 generic.go:356] "Generic (PLEG): container finished" podID="c9565699-e375-4d15-b61d-82d015b96298" containerID="5305a66390d8facbd26ce094250a124446b1161bedba03fe66a03e5b2bdd781e" exitCode=0 Jul 03 10:46:42.755836 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.755647 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" Jul 03 10:46:42.755836 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.755669 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" event={"ID":"c9565699-e375-4d15-b61d-82d015b96298","Type":"ContainerDied","Data":"5305a66390d8facbd26ce094250a124446b1161bedba03fe66a03e5b2bdd781e"} Jul 03 10:46:42.755836 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.755725 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk" event={"ID":"c9565699-e375-4d15-b61d-82d015b96298","Type":"ContainerDied","Data":"301eeb9087d5b42e327d24dc050bfbeb1e20625618bc1381424f2e5ce0739ed6"} Jul 03 10:46:42.755836 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.755778 2573 scope.go:117] "RemoveContainer" containerID="5305a66390d8facbd26ce094250a124446b1161bedba03fe66a03e5b2bdd781e" Jul 03 10:46:42.765125 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.765107 2573 scope.go:117] "RemoveContainer" containerID="5305a66390d8facbd26ce094250a124446b1161bedba03fe66a03e5b2bdd781e" Jul 03 10:46:42.765380 ip-10-0-134-191 kubenswrapper[2573]: E0703 10:46:42.765365 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5305a66390d8facbd26ce094250a124446b1161bedba03fe66a03e5b2bdd781e\": container with ID starting with 5305a66390d8facbd26ce094250a124446b1161bedba03fe66a03e5b2bdd781e not found: ID does not exist" containerID="5305a66390d8facbd26ce094250a124446b1161bedba03fe66a03e5b2bdd781e" Jul 03 10:46:42.765428 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.765387 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5305a66390d8facbd26ce094250a124446b1161bedba03fe66a03e5b2bdd781e"} err="failed to get container status \"5305a66390d8facbd26ce094250a124446b1161bedba03fe66a03e5b2bdd781e\": rpc error: code = NotFound desc = could not find container \"5305a66390d8facbd26ce094250a124446b1161bedba03fe66a03e5b2bdd781e\": container with ID starting with 5305a66390d8facbd26ce094250a124446b1161bedba03fe66a03e5b2bdd781e not found: ID does not exist" Jul 03 10:46:42.774059 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.774033 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk"] Jul 03 10:46:42.778275 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:42.778251 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-62c45e-576ffbd7cf-nwsbk"] Jul 03 10:46:44.563538 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:44.563498 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9565699-e375-4d15-b61d-82d015b96298" path="/var/lib/kubelet/pods/c9565699-e375-4d15-b61d-82d015b96298/volumes" Jul 03 10:46:53.459509 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:53.459479 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717910-4thq5_f55e7c56-27c2-4783-a582-5689689477fd/cleanup/0.log" Jul 03 10:46:53.567037 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:53.567003 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717925-d6276_8b46b71c-add7-4029-a328-98149707b8f2/cleanup/0.log" Jul 03 10:46:54.045818 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:54.045789 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-789547c8f-4pl9r_3c088a64-a749-482a-a253-596c1345f9cc/copy-manifests/0.log" Jul 03 10:46:54.054812 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:54.054765 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-789547c8f-4pl9r_3c088a64-a749-482a-a253-596c1345f9cc/manager/0.log" Jul 03 10:46:54.404005 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:54.403980 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-dnsb8_65ea69a1-53b0-42b0-a6fd-197ef6bc23f4/postgres/0.log" Jul 03 10:46:55.967977 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:55.967948 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-kxcwc_eefa695e-b755-4069-8637-2837891c55b5/manager/0.log" Jul 03 10:46:57.135911 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:57.135877 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b56494769-5khf6_6802d445-8d73-4ccd-8ce4-1b6b30d6708b/kube-auth-proxy/0.log" Jul 03 10:46:57.492479 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:57.492408 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-9c9cx_ce5fa191-9b07-47c6-a101-aa1a96f95f07/payload-pre-processing/0.log" Jul 03 10:46:57.624990 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:57.624959 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-dhnq7_a6bd5c00-e546-4115-95eb-802c8cc30ddd/payload-processing/0.log" Jul 03 10:46:58.071934 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:58.071904 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-wj5l2_6f6e1ffc-2fb8-4fcc-9938-097e44e6863b/main/0.log" Jul 03 10:46:58.295310 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:46:58.295269 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-mrh8x_b4a8f69d-ffe3-4ed9-8fd9-49b9b47e95b3/main/0.log" Jul 03 10:47:02.285377 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.285344 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dwvz5/must-gather-plq8x"] Jul 03 10:47:02.285811 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.285782 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="194e39df-559b-4320-a4d5-0cc0a3909ac3" containerName="cleanup" Jul 03 10:47:02.285811 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.285796 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="194e39df-559b-4320-a4d5-0cc0a3909ac3" containerName="cleanup" Jul 03 10:47:02.285811 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.285809 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0da2b8ef-5f9f-473f-b5e9-832f5564336b" containerName="main" Jul 03 10:47:02.285939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.285815 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0da2b8ef-5f9f-473f-b5e9-832f5564336b" containerName="main" Jul 03 10:47:02.285939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.285827 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="569893b6-194a-41e3-8bd0-9563864ea7ea" containerName="maas-api" Jul 03 10:47:02.285939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.285832 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="569893b6-194a-41e3-8bd0-9563864ea7ea" containerName="maas-api" Jul 03 10:47:02.285939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.285840 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9565699-e375-4d15-b61d-82d015b96298" containerName="maas-api" Jul 03 10:47:02.285939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.285845 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9565699-e375-4d15-b61d-82d015b96298" containerName="maas-api" Jul 03 10:47:02.285939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.285852 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62e2d2e5-6f7a-49a8-a32a-7227e8cf108b" containerName="cleanup" Jul 03 10:47:02.285939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.285858 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="62e2d2e5-6f7a-49a8-a32a-7227e8cf108b" containerName="cleanup" Jul 03 10:47:02.285939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.285867 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b46b71c-add7-4029-a328-98149707b8f2" containerName="cleanup" Jul 03 10:47:02.285939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.285872 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b46b71c-add7-4029-a328-98149707b8f2" containerName="cleanup" Jul 03 10:47:02.285939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.285928 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="569893b6-194a-41e3-8bd0-9563864ea7ea" containerName="maas-api" Jul 03 10:47:02.285939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.285934 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b46b71c-add7-4029-a328-98149707b8f2" containerName="cleanup" Jul 03 10:47:02.285939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.285940 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="194e39df-559b-4320-a4d5-0cc0a3909ac3" containerName="cleanup" Jul 03 10:47:02.285939 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.285946 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9565699-e375-4d15-b61d-82d015b96298" containerName="maas-api" Jul 03 10:47:02.286326 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.285952 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="62e2d2e5-6f7a-49a8-a32a-7227e8cf108b" containerName="cleanup" Jul 03 10:47:02.286326 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.285958 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0da2b8ef-5f9f-473f-b5e9-832f5564336b" containerName="main" Jul 03 10:47:02.289295 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.289273 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dwvz5/must-gather-plq8x" Jul 03 10:47:02.293229 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.293199 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dwvz5\"/\"kube-root-ca.crt\"" Jul 03 10:47:02.294658 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.294639 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dwvz5\"/\"default-dockercfg-zrqnd\"" Jul 03 10:47:02.294805 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.294656 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dwvz5\"/\"openshift-service-ca.crt\"" Jul 03 10:47:02.299148 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.299121 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dwvz5/must-gather-plq8x"] Jul 03 10:47:02.392096 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.392058 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nkjw\" (UniqueName: \"kubernetes.io/projected/12eb4bed-1c4d-4c23-9bb8-70c581d6134e-kube-api-access-8nkjw\") pod \"must-gather-plq8x\" (UID: \"12eb4bed-1c4d-4c23-9bb8-70c581d6134e\") " pod="openshift-must-gather-dwvz5/must-gather-plq8x" Jul 03 10:47:02.392252 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.392113 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/12eb4bed-1c4d-4c23-9bb8-70c581d6134e-must-gather-output\") pod \"must-gather-plq8x\" (UID: \"12eb4bed-1c4d-4c23-9bb8-70c581d6134e\") " pod="openshift-must-gather-dwvz5/must-gather-plq8x" Jul 03 10:47:02.493029 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.492994 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nkjw\" (UniqueName: \"kubernetes.io/projected/12eb4bed-1c4d-4c23-9bb8-70c581d6134e-kube-api-access-8nkjw\") pod \"must-gather-plq8x\" (UID: \"12eb4bed-1c4d-4c23-9bb8-70c581d6134e\") " pod="openshift-must-gather-dwvz5/must-gather-plq8x" Jul 03 10:47:02.493202 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.493043 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/12eb4bed-1c4d-4c23-9bb8-70c581d6134e-must-gather-output\") pod \"must-gather-plq8x\" (UID: \"12eb4bed-1c4d-4c23-9bb8-70c581d6134e\") " pod="openshift-must-gather-dwvz5/must-gather-plq8x" Jul 03 10:47:02.493424 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.493404 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/12eb4bed-1c4d-4c23-9bb8-70c581d6134e-must-gather-output\") pod \"must-gather-plq8x\" (UID: \"12eb4bed-1c4d-4c23-9bb8-70c581d6134e\") " pod="openshift-must-gather-dwvz5/must-gather-plq8x" Jul 03 10:47:02.501621 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.501587 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nkjw\" (UniqueName: \"kubernetes.io/projected/12eb4bed-1c4d-4c23-9bb8-70c581d6134e-kube-api-access-8nkjw\") pod \"must-gather-plq8x\" (UID: \"12eb4bed-1c4d-4c23-9bb8-70c581d6134e\") " pod="openshift-must-gather-dwvz5/must-gather-plq8x" Jul 03 10:47:02.600136 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.600049 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dwvz5/must-gather-plq8x" Jul 03 10:47:02.740904 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.740877 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dwvz5/must-gather-plq8x"] Jul 03 10:47:02.743542 ip-10-0-134-191 kubenswrapper[2573]: W0703 10:47:02.743517 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12eb4bed_1c4d_4c23_9bb8_70c581d6134e.slice/crio-a3c90618bbb474b16d0d57d049b3170a4d5c9b8a93ef5e92fc168d88fbde3b1e WatchSource:0}: Error finding container a3c90618bbb474b16d0d57d049b3170a4d5c9b8a93ef5e92fc168d88fbde3b1e: Status 404 returned error can't find the container with id a3c90618bbb474b16d0d57d049b3170a4d5c9b8a93ef5e92fc168d88fbde3b1e Jul 03 10:47:02.748565 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.748547 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 03 10:47:02.858426 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:02.858345 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dwvz5/must-gather-plq8x" event={"ID":"12eb4bed-1c4d-4c23-9bb8-70c581d6134e","Type":"ContainerStarted","Data":"a3c90618bbb474b16d0d57d049b3170a4d5c9b8a93ef5e92fc168d88fbde3b1e"} Jul 03 10:47:03.867720 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:03.867665 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dwvz5/must-gather-plq8x" event={"ID":"12eb4bed-1c4d-4c23-9bb8-70c581d6134e","Type":"ContainerStarted","Data":"daf9febd61b2f80613afa3ab8454a591e25489aab88f1b3789d8fba4aefa2f41"} Jul 03 10:47:04.879955 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:04.879906 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dwvz5/must-gather-plq8x" event={"ID":"12eb4bed-1c4d-4c23-9bb8-70c581d6134e","Type":"ContainerStarted","Data":"23617893cd59bd9091d8d422ebb33579b6f6213c781dba0ef175c871c2f808df"} Jul 03 10:47:04.899174 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:04.899112 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dwvz5/must-gather-plq8x" podStartSLOduration=1.9107582889999999 podStartE2EDuration="2.899091662s" totalImagesPullingTime="988.333373ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-03 10:47:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:47:04.897948406 +0000 UTC m=+2479.809837712" watchObservedRunningTime="2026-07-03 10:47:04.899091662 +0000 UTC m=+2479.810980965" Jul 03 10:47:05.348870 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:05.348833 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qqgm8_e8ff375c-d36d-431c-ac6e-c2aef1d3c26a/global-pull-secret-syncer/0.log" Jul 03 10:47:05.520869 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:05.520837 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-s7jq9_3c7a6ee8-12fe-4c0e-a0b2-e3b828f94858/konnectivity-agent/0.log" Jul 03 10:47:05.550495 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:05.550459 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-191.ec2.internal_0928399de55edb216381d354ba304559/haproxy/0.log" Jul 03 10:47:10.257791 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:10.257694 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-kxcwc_eefa695e-b755-4069-8637-2837891c55b5/manager/0.log" Jul 03 10:47:12.632238 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:12.632194 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c3dca71e-192e-4cf2-932b-a8e878872f70/alertmanager/0.log" Jul 03 10:47:12.656534 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:12.656501 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c3dca71e-192e-4cf2-932b-a8e878872f70/config-reloader/0.log" Jul 03 10:47:12.684529 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:12.684447 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c3dca71e-192e-4cf2-932b-a8e878872f70/kube-rbac-proxy-web/0.log" Jul 03 10:47:12.713440 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:12.713389 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c3dca71e-192e-4cf2-932b-a8e878872f70/kube-rbac-proxy/0.log" Jul 03 10:47:12.737890 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:12.737857 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c3dca71e-192e-4cf2-932b-a8e878872f70/kube-rbac-proxy-metric/0.log" Jul 03 10:47:12.762845 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:12.762803 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c3dca71e-192e-4cf2-932b-a8e878872f70/prom-label-proxy/0.log" Jul 03 10:47:12.784946 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:12.784919 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c3dca71e-192e-4cf2-932b-a8e878872f70/init-config-reloader/0.log" Jul 03 10:47:12.866412 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:12.866385 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-vr6b4_7b2ebc75-c820-4e81-9704-e2580f24da82/kube-state-metrics/0.log" Jul 03 10:47:12.886676 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:12.886646 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-vr6b4_7b2ebc75-c820-4e81-9704-e2580f24da82/kube-rbac-proxy-main/0.log" Jul 03 10:47:12.915357 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:12.915318 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-vr6b4_7b2ebc75-c820-4e81-9704-e2580f24da82/kube-rbac-proxy-self/0.log" Jul 03 10:47:12.950697 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:12.950614 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-865d4c69d7-pppdd_67615ad3-a3cc-45e9-a148-c0476d653e11/metrics-server/0.log" Jul 03 10:47:12.988823 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:12.988794 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-ncl65_eeccf333-5485-44f1-b0c6-617be538c81b/monitoring-plugin/0.log" Jul 03 10:47:13.179029 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.178993 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gfjrs_cdeaefc1-cb0f-4c41-8e6d-14918dfe9318/node-exporter/0.log" Jul 03 10:47:13.206979 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.206882 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gfjrs_cdeaefc1-cb0f-4c41-8e6d-14918dfe9318/kube-rbac-proxy/0.log" Jul 03 10:47:13.230230 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.230091 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gfjrs_cdeaefc1-cb0f-4c41-8e6d-14918dfe9318/init-textfile/0.log" Jul 03 10:47:13.258643 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.258611 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-7gn6n_8f73dc61-257e-4a34-80ef-8c86c5d4511e/kube-rbac-proxy-main/0.log" Jul 03 10:47:13.288351 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.288314 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-7gn6n_8f73dc61-257e-4a34-80ef-8c86c5d4511e/kube-rbac-proxy-self/0.log" Jul 03 10:47:13.310162 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.310137 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-7gn6n_8f73dc61-257e-4a34-80ef-8c86c5d4511e/openshift-state-metrics/0.log" Jul 03 10:47:13.347792 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.347710 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_47b2a6a3-c715-4a87-b570-517b831398c0/prometheus/0.log" Jul 03 10:47:13.370332 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.370299 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_47b2a6a3-c715-4a87-b570-517b831398c0/config-reloader/0.log" Jul 03 10:47:13.394690 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.394651 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_47b2a6a3-c715-4a87-b570-517b831398c0/thanos-sidecar/0.log" Jul 03 10:47:13.434848 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.434817 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_47b2a6a3-c715-4a87-b570-517b831398c0/kube-rbac-proxy-web/0.log" Jul 03 10:47:13.463134 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.463047 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_47b2a6a3-c715-4a87-b570-517b831398c0/kube-rbac-proxy/0.log" Jul 03 10:47:13.486379 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.486351 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_47b2a6a3-c715-4a87-b570-517b831398c0/kube-rbac-proxy-thanos/0.log" Jul 03 10:47:13.507005 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.506971 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_47b2a6a3-c715-4a87-b570-517b831398c0/init-config-reloader/0.log" Jul 03 10:47:13.614618 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.614589 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-754fffd6f7-2n7px_92777b98-f96f-4bb5-927f-5a4b46356a0c/telemeter-client/0.log" Jul 03 10:47:13.642243 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.642210 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-754fffd6f7-2n7px_92777b98-f96f-4bb5-927f-5a4b46356a0c/reload/0.log" Jul 03 10:47:13.670090 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.670058 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-754fffd6f7-2n7px_92777b98-f96f-4bb5-927f-5a4b46356a0c/kube-rbac-proxy/0.log" Jul 03 10:47:13.700571 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.700548 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f8d657f7c-j76pt_92357b58-c179-4495-be4f-295b61cc0ad5/thanos-query/0.log" Jul 03 10:47:13.728923 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.728816 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f8d657f7c-j76pt_92357b58-c179-4495-be4f-295b61cc0ad5/kube-rbac-proxy-web/0.log" Jul 03 10:47:13.755328 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.755293 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f8d657f7c-j76pt_92357b58-c179-4495-be4f-295b61cc0ad5/kube-rbac-proxy/0.log" Jul 03 10:47:13.778830 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.778804 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f8d657f7c-j76pt_92357b58-c179-4495-be4f-295b61cc0ad5/prom-label-proxy/0.log" Jul 03 10:47:13.809025 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.808994 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f8d657f7c-j76pt_92357b58-c179-4495-be4f-295b61cc0ad5/kube-rbac-proxy-rules/0.log" Jul 03 10:47:13.838546 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:13.838517 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f8d657f7c-j76pt_92357b58-c179-4495-be4f-295b61cc0ad5/kube-rbac-proxy-metrics/0.log" Jul 03 10:47:15.365107 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.365059 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4"] Jul 03 10:47:15.371928 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.371896 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" Jul 03 10:47:15.377976 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.377943 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4"] Jul 03 10:47:15.540556 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.540517 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/693693ce-7b94-4e5f-9339-d223f4ef5cc7-lib-modules\") pod \"perf-node-gather-daemonset-sgmh4\" (UID: \"693693ce-7b94-4e5f-9339-d223f4ef5cc7\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" Jul 03 10:47:15.540803 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.540614 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/693693ce-7b94-4e5f-9339-d223f4ef5cc7-proc\") pod \"perf-node-gather-daemonset-sgmh4\" (UID: \"693693ce-7b94-4e5f-9339-d223f4ef5cc7\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" Jul 03 10:47:15.540803 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.540646 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vx5v\" (UniqueName: \"kubernetes.io/projected/693693ce-7b94-4e5f-9339-d223f4ef5cc7-kube-api-access-2vx5v\") pod \"perf-node-gather-daemonset-sgmh4\" (UID: \"693693ce-7b94-4e5f-9339-d223f4ef5cc7\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" Jul 03 10:47:15.540803 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.540721 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/693693ce-7b94-4e5f-9339-d223f4ef5cc7-sys\") pod \"perf-node-gather-daemonset-sgmh4\" (UID: \"693693ce-7b94-4e5f-9339-d223f4ef5cc7\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" Jul 03 10:47:15.540803 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.540782 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/693693ce-7b94-4e5f-9339-d223f4ef5cc7-podres\") pod \"perf-node-gather-daemonset-sgmh4\" (UID: \"693693ce-7b94-4e5f-9339-d223f4ef5cc7\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" Jul 03 10:47:15.642369 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.642322 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/693693ce-7b94-4e5f-9339-d223f4ef5cc7-sys\") pod \"perf-node-gather-daemonset-sgmh4\" (UID: \"693693ce-7b94-4e5f-9339-d223f4ef5cc7\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" Jul 03 10:47:15.642571 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.642410 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/693693ce-7b94-4e5f-9339-d223f4ef5cc7-podres\") pod \"perf-node-gather-daemonset-sgmh4\" (UID: \"693693ce-7b94-4e5f-9339-d223f4ef5cc7\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" Jul 03 10:47:15.642571 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.642502 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/693693ce-7b94-4e5f-9339-d223f4ef5cc7-lib-modules\") pod \"perf-node-gather-daemonset-sgmh4\" (UID: \"693693ce-7b94-4e5f-9339-d223f4ef5cc7\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" Jul 03 10:47:15.642571 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.642560 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/693693ce-7b94-4e5f-9339-d223f4ef5cc7-proc\") pod \"perf-node-gather-daemonset-sgmh4\" (UID: \"693693ce-7b94-4e5f-9339-d223f4ef5cc7\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" Jul 03 10:47:15.642760 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.642590 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vx5v\" (UniqueName: \"kubernetes.io/projected/693693ce-7b94-4e5f-9339-d223f4ef5cc7-kube-api-access-2vx5v\") pod \"perf-node-gather-daemonset-sgmh4\" (UID: \"693693ce-7b94-4e5f-9339-d223f4ef5cc7\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" Jul 03 10:47:15.643056 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.643029 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/693693ce-7b94-4e5f-9339-d223f4ef5cc7-sys\") pod \"perf-node-gather-daemonset-sgmh4\" (UID: \"693693ce-7b94-4e5f-9339-d223f4ef5cc7\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" Jul 03 10:47:15.643415 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.643392 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/693693ce-7b94-4e5f-9339-d223f4ef5cc7-podres\") pod \"perf-node-gather-daemonset-sgmh4\" (UID: \"693693ce-7b94-4e5f-9339-d223f4ef5cc7\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" Jul 03 10:47:15.643854 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.643832 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/693693ce-7b94-4e5f-9339-d223f4ef5cc7-lib-modules\") pod \"perf-node-gather-daemonset-sgmh4\" (UID: \"693693ce-7b94-4e5f-9339-d223f4ef5cc7\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" Jul 03 10:47:15.643961 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.643893 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/693693ce-7b94-4e5f-9339-d223f4ef5cc7-proc\") pod \"perf-node-gather-daemonset-sgmh4\" (UID: \"693693ce-7b94-4e5f-9339-d223f4ef5cc7\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" Jul 03 10:47:15.652324 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.652290 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vx5v\" (UniqueName: \"kubernetes.io/projected/693693ce-7b94-4e5f-9339-d223f4ef5cc7-kube-api-access-2vx5v\") pod \"perf-node-gather-daemonset-sgmh4\" (UID: \"693693ce-7b94-4e5f-9339-d223f4ef5cc7\") " pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" Jul 03 10:47:15.687803 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.687769 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" Jul 03 10:47:15.897726 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.897301 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4"] Jul 03 10:47:15.966801 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:15.966710 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" event={"ID":"693693ce-7b94-4e5f-9339-d223f4ef5cc7","Type":"ContainerStarted","Data":"0fa980f4850bd616a5f44e10926409d08748d9c07d5ee982e004d15c79566502"} Jul 03 10:47:16.197623 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:16.197554 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6846446956-mzffw_c55b2578-0178-406c-8534-3279badb97c0/console/0.log" Jul 03 10:47:16.974280 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:16.974245 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" event={"ID":"693693ce-7b94-4e5f-9339-d223f4ef5cc7","Type":"ContainerStarted","Data":"0926050da7c3d254dfbd4dd58ed6d2c2eecf88b41c2a956a61f988b9d6b99638"} Jul 03 10:47:16.974721 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:16.974331 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" Jul 03 10:47:16.993065 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:16.992991 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" podStartSLOduration=1.9929669479999998 podStartE2EDuration="1.992966948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-03 10:47:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-03 10:47:16.990239066 +0000 UTC m=+2491.902128370" watchObservedRunningTime="2026-07-03 10:47:16.992966948 +0000 UTC m=+2491.904856256" Jul 03 10:47:17.991763 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:17.991712 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x27kq_0d094a5f-57d0-4fec-8a73-334d94229e78/dns/0.log" Jul 03 10:47:18.015808 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:18.015780 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x27kq_0d094a5f-57d0-4fec-8a73-334d94229e78/kube-rbac-proxy/0.log" Jul 03 10:47:18.065614 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:18.065586 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hpxfb_8457043a-d27e-4962-baaa-cebc19fc88dd/dns-node-resolver/0.log" Jul 03 10:47:18.611175 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:18.611150 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2gnl9_7fcf8890-a36d-404f-95e1-631a6aa5c137/node-ca/0.log" Jul 03 10:47:19.815147 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:19.815115 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b56494769-5khf6_6802d445-8d73-4ccd-8ce4-1b6b30d6708b/kube-auth-proxy/0.log" Jul 03 10:47:20.028794 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:20.028764 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-9c9cx_ce5fa191-9b07-47c6-a101-aa1a96f95f07/payload-pre-processing/0.log" Jul 03 10:47:20.089256 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:20.089160 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-dhnq7_a6bd5c00-e546-4115-95eb-802c8cc30ddd/payload-processing/0.log" Jul 03 10:47:20.894230 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:20.894202 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sbmfl_4818679d-6a2e-4a1d-816b-9a52a442430a/serve-healthcheck-canary/0.log" Jul 03 10:47:21.499261 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:21.499225 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bkrdf_ebb4964b-254e-407e-b5b1-889066572d2d/kube-rbac-proxy/0.log" Jul 03 10:47:21.521515 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:21.521484 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bkrdf_ebb4964b-254e-407e-b5b1-889066572d2d/exporter/0.log" Jul 03 10:47:21.541134 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:21.541103 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bkrdf_ebb4964b-254e-407e-b5b1-889066572d2d/extractor/0.log" Jul 03 10:47:22.992407 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:22.992376 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dwvz5/perf-node-gather-daemonset-sgmh4" Jul 03 10:47:23.948806 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:23.948772 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717910-4thq5_f55e7c56-27c2-4783-a582-5689689477fd/cleanup/0.log" Jul 03 10:47:23.975242 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:23.975214 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717925-d6276_8b46b71c-add7-4029-a328-98149707b8f2/cleanup/0.log" Jul 03 10:47:24.233078 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:24.232950 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-789547c8f-4pl9r_3c088a64-a749-482a-a253-596c1345f9cc/manager/0.log" Jul 03 10:47:24.258447 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:24.258320 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-789547c8f-4pl9r_3c088a64-a749-482a-a253-596c1345f9cc/copy-manifests/0.log" Jul 03 10:47:24.421170 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:24.421146 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-dnsb8_65ea69a1-53b0-42b0-a6fd-197ef6bc23f4/postgres/0.log" Jul 03 10:47:25.895640 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:25.895608 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5779fb96bf-qkbc9_a261ae40-ff24-43d2-ae42-1f8cb65c2304/manager/0.log" Jul 03 10:47:25.946931 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:25.946883 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-g24pn_f31e0b7b-d1ad-4586-a9ff-e1d7d2e2b5ae/openshift-lws-operator/0.log" Jul 03 10:47:26.120945 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:26.120916 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b56494769-5khf6_6802d445-8d73-4ccd-8ce4-1b6b30d6708b/kube-auth-proxy/0.log" Jul 03 10:47:26.235008 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:26.234928 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-9c9cx_ce5fa191-9b07-47c6-a101-aa1a96f95f07/payload-pre-processing/0.log" Jul 03 10:47:26.289223 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:26.289188 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-dhnq7_a6bd5c00-e546-4115-95eb-802c8cc30ddd/payload-processing/0.log" Jul 03 10:47:27.688632 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:27.688556 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b56494769-5khf6_6802d445-8d73-4ccd-8ce4-1b6b30d6708b/kube-auth-proxy/0.log" Jul 03 10:47:27.801955 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:27.801925 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-9c9cx_ce5fa191-9b07-47c6-a101-aa1a96f95f07/payload-pre-processing/0.log" Jul 03 10:47:27.845919 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:27.845883 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-dhnq7_a6bd5c00-e546-4115-95eb-802c8cc30ddd/payload-processing/0.log" Jul 03 10:47:32.212226 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:32.212198 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62lv9_4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e/kube-multus-additional-cni-plugins/0.log" Jul 03 10:47:32.233305 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:32.233279 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62lv9_4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e/egress-router-binary-copy/0.log" Jul 03 10:47:32.254322 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:32.254290 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62lv9_4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e/cni-plugins/0.log" Jul 03 10:47:32.274540 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:32.274510 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62lv9_4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e/bond-cni-plugin/0.log" Jul 03 10:47:32.297226 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:32.297197 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62lv9_4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e/routeoverride-cni/0.log" Jul 03 10:47:32.318023 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:32.317993 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62lv9_4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e/whereabouts-cni-bincopy/0.log" Jul 03 10:47:32.337815 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:32.337791 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-62lv9_4b8cd90a-dd75-4ddc-b0cb-a4d4573bd38e/whereabouts-cni/0.log" Jul 03 10:47:32.747699 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:32.747670 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-tjzjw_0238c2d7-0223-4bbd-bc30-cb8c8901f737/kube-multus/0.log" Jul 03 10:47:32.814621 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:32.814592 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ffkpf_a077a8e0-9998-4603-9c7c-f92f296c2dbf/network-metrics-daemon/0.log" Jul 03 10:47:32.833946 ip-10-0-134-191 kubenswrapper[2573]: I0703 10:47:32.833853 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ffkpf_a077a8e0-9998-4603-9c7c-f92f296c2dbf/kube-rbac-proxy/0.log"